-- Logs begin at Sat 2022-09-24 14:21:23 UTC, end at Sat 2022-09-24 15:19:18 UTC. -- Sep 24 14:45:28.749723 dv-cinder-859194-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 24 14:45:28.756965 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 24 14:45:28 2022] *** Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: nodename: dv-cinder-859194-1-iscsi Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: machine: x86_64 Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: clock source: unix Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: pcre jit disabled Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: detected number of CPU cores: 4 Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: current working directory: / Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: detected binary path: /usr/bin/uwsgi-core Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: your processes number limit is 47680 Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: your memory page size is 4096 bytes Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: detected max file descriptor number: 2048 Sep 24 14:45:28.760133 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: lock engine: pthread robust mutexes Sep 24 14:45:28.766428 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: thunder lock: enabled Sep 24 14:45:28.766428 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 24 14:45:28.766428 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 24 14:45:28.800107 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: Python main interpreter initialized at 0x55f6c80cf9c0 Sep 24 14:45:28.800107 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: python threads support enabled Sep 24 14:45:28.800107 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: your server socket listen backlog is limited to 100 connections Sep 24 14:45:28.800107 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: your mercy for graceful operations on workers is 90 seconds Sep 24 14:45:28.800425 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: mapped 403077 bytes (393 KB) for 2 cores Sep 24 14:45:28.800536 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: *** Operational MODE: preforking *** Sep 24 14:45:28.800810 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: *** uWSGI is running in multiple interpreter mode *** Sep 24 14:45:28.800810 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: spawned uWSGI master process (pid: 81391) Sep 24 14:45:28.801011 dv-cinder-859194-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 24 14:45:28.805932 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: spawned uWSGI worker 1 (pid: 81402, cores: 1) Sep 24 14:45:28.805932 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: spawned uWSGI worker 2 (pid: 81403, cores: 1) Sep 24 14:45:28.805932 dv-cinder-859194-1-iscsi devstack@keystone.service[81391]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 24 14:45:33.092566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:45:33.092566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: from cryptography.utils import int_from_bytes Sep 24 14:45:33.093359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:45:33.093359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: from cryptography.utils import int_from_bytes Sep 24 14:45:33.095849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:45:33.095849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Sep 24 14:45:33.096309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:45:33.096309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Sep 24 14:45:33.143033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 14:45:33.143224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 14:45:33.143416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] command line args: [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 14:45:33.143687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 14:45:33.143867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ================================================================================ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 14:45:33.144124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] admin_token = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.144374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config_dir = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.144562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.144744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config_source = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.144926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] debug = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.145110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 14:45:33.145189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.145383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 14:45:33.145508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] command line args: [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 14:45:33.145589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.145651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.145762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 14:45:33.145916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ================================================================================ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 14:45:33.146171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_dir = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_config_append = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_source = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.146988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_dir = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] debug = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147889 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.147966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_config_append = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.148634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_dir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.149990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.150974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] publish_errors = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.151910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.152950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] standard_threads = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] publish_errors = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_journal = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.153981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_json = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_stderr = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_syslog = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.154971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] standard_threads = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.155089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.155203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.155316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.155427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.155546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.155668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.155668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_journal = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.155887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.156002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_json = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.156114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.156226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_stderr = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.156226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_syslog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.156450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.156568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:45:33.156682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.156795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.156914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.157983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.158914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.159993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.160990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.161851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.162938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.163944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.164065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.164180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.164292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.165974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.166992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.167931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.168952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.169868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.170988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171142 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.171855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.172983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.173921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.174984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.175889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.176951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177180 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.177871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.178931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.179874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.180942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.external = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.181868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.password = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.token = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.external = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.182999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.password = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.183948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.token = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184303 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184776 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.184890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.185955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.186910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.187972 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188677 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.188914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.189961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.190908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.191998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.192900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.193996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.194934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.195912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.196987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.197940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.198894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.199949 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.200897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.201750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.202964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.203889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.204829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.205953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.206899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.207946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.208996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.209977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.210932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.211046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.211161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.211834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.211954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212677 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.212919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.213980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.214927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.215991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.216926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.217870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.218973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.219895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.220926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.221897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.222963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.223999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.224917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.225843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.226931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.227555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.227676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.227791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.227907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.228878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.229980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.230904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.231960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.232907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.233023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 14:45:33.233138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.233138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.233138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.233138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.233138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:45:33.233138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 14:45:33.259081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.259311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.259410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.259538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.259623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.259782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.267851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.268022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.296701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.296818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.296945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.297024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.352199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.357051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.401581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.401810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.401956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.402193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.557916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.558056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.558150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.558271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.581526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.581750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.581966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.582936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.583070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.583286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.594550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.595115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.607997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.608859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:45:33.625174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.625384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:45:33.625611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.625726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.625808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:45:33.625950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:45:33.626177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.626262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.626379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:45:33.626460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.626658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.626747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.626927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.627008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.627191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.627450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.637003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.637279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.637549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.637632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.637830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.637934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.638153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.638441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.644773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.645372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.647676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.647962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.648243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.648323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.648529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.648611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.648865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.649440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.649620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.649890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.649983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.650223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.650384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.650496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.650772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.650850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.651040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.651173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.651598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.651913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.652197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.652473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.675071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.676546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.676833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.678112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.679589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.679874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.681529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.681819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.682129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.684546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.684831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.685114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.688452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.688740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.691427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.691711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.694501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.694801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.695094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.695386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.696198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.696483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.696763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.697038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.697989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.698289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.698576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.698859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.699132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.699409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.699680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.699950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.712964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.713245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.715852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.716127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.717064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.719968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.720254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.720551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.723199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.723497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.724704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.727723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.727850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.730903 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.731105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.731408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.731699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.731992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.732271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.732559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.734094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.734392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.734676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.734958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.735234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.735529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.748526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.748817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.749091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.749360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.751426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.751712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.751979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.752240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.756202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.757754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.759574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.759844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.760122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.761098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.762653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.762854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.762936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.763139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.763430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.763718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.764107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.765581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.765865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.766172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.766456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.766851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.775743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.776031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.776315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.776595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.776975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.777265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.777547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.778442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.778727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.779008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.779287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.779668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.779955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.780233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.792819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.795478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.795761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.798377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.798671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.800126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.801229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.801589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.801876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.802297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.802771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.804242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.804524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.804918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.809211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.810662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.811761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.813224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.813530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.813946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.814237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.814513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.816107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.816501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.816783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.817057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.822856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.823149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.823441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.825377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.825668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.825973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.830306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.830592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.830871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.831147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.831420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.831695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.831973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.832266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.832538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.832771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:45:33.832851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.833877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.834023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.834179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.834451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.834726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.835007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.835279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.835557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.835814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.836075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.836370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:45:33.868367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f6c80cf9c0 pid: 81402 (default app) Sep 24 14:45:33.870591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f6c80cf9c0 pid: 81403 (default app) Sep 24 14:45:33.871631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c38a-cd6c-46b5-9595-07c82638c3e2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:33.871920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c38a-cd6c-46b5-9595-07c82638c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:33.872196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c38a-cd6c-46b5-9595-07c82638c3e2 None None] PATH_INFO: `/v3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:33.876370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Sat Sep 24 14:45:33 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:40.619371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:40.619878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:40.619978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:40.624238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:45:40.629463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:40.630027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:40.630118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:40.701126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: INFO dbcounter [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] Registered counter for database keystone Sep 24 14:45:40.720074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG oslo_db.sqlalchemy.engines [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81402) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 14:45:40.726212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writer thread running {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 14:45:40.906068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG passlib.handlers.bcrypt [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81402) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 14:45:40.906380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG passlib.handlers.bcrypt [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81402) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 14:45:40.964542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:40.966169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:41.093342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 14:45:41.536639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:41.607636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: INFO dbcounter [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] Registered counter for database keystone Sep 24 14:45:41.629938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG oslo_db.sqlalchemy.engines [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81403) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 14:45:41.636753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writer thread running {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 14:45:41.755833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f973c-e00e-48c4-a11b-3b04eeefb085 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:41.760707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:41.761328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:41.761612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:41.761889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:41.770091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:41.821968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING keystone.common.rbac_enforcer.enforcer [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 14:45:41.822260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f84ac-62fd-41b5-bd5c-3582de0e08a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:41.831789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Sat Sep 24 14:45:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 14:45:41.838455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:41.855052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2158c09-fde1-4ccc-8be4-cc4aaf384541 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:41.859522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:41.860635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:41.860925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:41.861203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] PATH_INFO: `/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:41.863063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] RBAC: Authorizing `identity:get_project(project_id=54d7e41cbbc346c5a0a397fdcad9ecdd)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:41.916729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING keystone.common.rbac_enforcer.enforcer [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 14:45:41.917081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f03b8-e331-4041-a6f0-575a6c977203 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:41.921685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Sat Sep 24 14:45:41 2022] GET /identity/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:44.044179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1723-1b0c-4b4f-8a33-ca7bf4f8c231 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:44.044728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1723-1b0c-4b4f-8a33-ca7bf4f8c231 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:44.044728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1723-1b0c-4b4f-8a33-ca7bf4f8c231 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:44.048855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:44.054219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:44.055223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:44.055360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:44.074898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:44.076397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:44.084101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 24 14:45:44.347171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1723-1b0c-4b4f-8a33-ca7bf4f8c231 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:44.359557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1723-1b0c-4b4f-8a33-ca7bf4f8c231 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:44.363951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3c230-5731-44b7-9111-bf89c2ee4943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:44.364535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3c230-5731-44b7-9111-bf89c2ee4943 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:44.364816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3c230-5731-44b7-9111-bf89c2ee4943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:44.365092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3c230-5731-44b7-9111-bf89c2ee4943 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:44.372557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3c230-5731-44b7-9111-bf89c2ee4943 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:44.374346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3c230-5731-44b7-9111-bf89c2ee4943 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:44.382875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Sat Sep 24 14:45:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 14:45:44.387945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:44.396968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada19b4-f181-477f-a977-47ff4c8f04bb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:44.401814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1458f-1249-4503-9708-5dca8a72a34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:44.402467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1458f-1249-4503-9708-5dca8a72a34d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:44.402752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1458f-1249-4503-9708-5dca8a72a34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:44.403029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1458f-1249-4503-9708-5dca8a72a34d admin admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:44.404334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1458f-1249-4503-9708-5dca8a72a34d admin admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:44.406211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1458f-1249-4503-9708-5dca8a72a34d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:44.407728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Sat Sep 24 14:45:44 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:46.610391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8419215-7b3e-496e-8003-4a16a695756a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:46.610925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8419215-7b3e-496e-8003-4a16a695756a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:46.610925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8419215-7b3e-496e-8003-4a16a695756a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:46.612275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:46.618130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:46.618544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:46.618747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:46.637344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:46.638655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:46.652023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062eaee-de2d-4155-9c85-71810977856c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:46.652313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062eaee-de2d-4155-9c85-71810977856c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:46.652599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062eaee-de2d-4155-9c85-71810977856c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:46.653941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:45:46.658926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:46.659213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:46.659492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:46.701375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81403) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 14:45:46.701772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81403) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 14:45:46.718137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 6/12] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:45:46.750748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:46.752330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:46.851324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 7/13] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:45:46.985474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.010099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.090144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8ffef-3395-4d00-a69b-c612d5eb9c4c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.114865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.126479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.127113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.127390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.127662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.138590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:45:47.147365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:45:47.148521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:45:47.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:45:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:45:47.154294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.217835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82215b-072f-473d-bce8-d0bea3e259fd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.220112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.233722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.252979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697dea7-bd14-4b4d-b0a8-3628a7883ba5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.256250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.257107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.257384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.257651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.286131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.286729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.287016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.287294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.287961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:47.288267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.290238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.301767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 8/15] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:45:47.307460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.323928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.325461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.404973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5d20e-aab7-406f-89ad-5c06fd623fe5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.424393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 8/16] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:45:47.430044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.432655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.438731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.439371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.439655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.439930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.441070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.442572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.444136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:45:47.449349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.464104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.528088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283ddbd-5f79-4f96-954e-11a0012bba2f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.547763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39d62-8c8c-4ca6-99d8-4da2e4b8730a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.561520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.562408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.562696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.562974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.581681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.582344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.582439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.582625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.582895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.583810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.583998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.585444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.587554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 10/18] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:47.593349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:45:47.603217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:45:47.604128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:45:47.604924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:45:47.604924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:45:47.604924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:45:47.604924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 9/19] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:45:47.609882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:47.670919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.677348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:47.696410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23515a11-c9a8-4ee3-88f7-21291bdcca4b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.717150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c85f28-2021-4aa7-b706-ed748b76d4da None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:47.729240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.729852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.730152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.730432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.731147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.domain.id=default, user.id=3ea61b3146144581bb15be72dcc836fe) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:47.731454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.732893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752bb17-9341-452e-be40-c493d6a912ea None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.739647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 11/20] 10.222.0.61 () {56 vars in 1322 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/role_assignments?user.id=3ea61b3146144581bb15be72dcc836fe&scope.domain.id=default&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:45:47.750108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:47.750752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:47.751034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:47.751305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:47.751974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:47.752303 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:47.753727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f3de-8671-4a47-b90f-e1f6c565cfe9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:47.774404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 10/21] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:45:50.163492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f692-15c1-4aea-9822-3aae1013cc97 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:50.164055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f692-15c1-4aea-9822-3aae1013cc97 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:50.164055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f692-15c1-4aea-9822-3aae1013cc97 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:50.165698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 12/22] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:50.175222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9280d3-4ad1-4faa-8049-a8bbf9fffd79 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:50.175639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9280d3-4ad1-4faa-8049-a8bbf9fffd79 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:50.175810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9280d3-4ad1-4faa-8049-a8bbf9fffd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:50.202898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e9280d3-4ad1-4faa-8049-a8bbf9fffd79 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:50.204026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e9280d3-4ad1-4faa-8049-a8bbf9fffd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:50.293424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 11/23] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:45:50.642501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f359-5414-4b98-9934-447bf6954996 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:50.643207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f359-5414-4b98-9934-447bf6954996 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:50.643716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f359-5414-4b98-9934-447bf6954996 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:50.645549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 13/24] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:50.651241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6233-81d3-49c5-92e2-9e8a93e9336e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:50.651764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6233-81d3-49c5-92e2-9e8a93e9336e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:50.652258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6233-81d3-49c5-92e2-9e8a93e9336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:50.674884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-52ab6233-81d3-49c5-92e2-9e8a93e9336e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:50.676101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-52ab6233-81d3-49c5-92e2-9e8a93e9336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:50.733341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:50.747187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f359-5414-4b98-9934-447bf6954996 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:50.765070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 12/25] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:45:50.770210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d3d2-4e2f-48ab-9524-1db81855c3bf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:50.770518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d3d2-4e2f-48ab-9524-1db81855c3bf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:50.770863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d3d2-4e2f-48ab-9524-1db81855c3bf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:50.772136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 13/26] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:50.780898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:50.781192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:50.781473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:50.814943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:50.816175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:50.903338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 14/27] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:45:50.967085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f359-5414-4b98-9934-447bf6954996 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.024574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.025554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.026091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.026583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.042938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:45:51.048450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:45:51.049271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:45:51.050048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 14/28] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:45:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:45:51.057605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.184171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.202272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:51.214834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f96b17-c2b1-44c1-a909-93d76c9b0ddf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.228009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:51.295346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.296110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.296397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.296672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.297361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:51.297671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:51.299109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:51.315651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 15/29] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:45:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:45:51.316787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.355608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68730cf5-1b7e-4e00-8c69-5a8a0527edc3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.391908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.392781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.393331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.393750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.394694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:51.396249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:51.438566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bc11a-93e0-4b87-a2d3-cc4d42c52dd5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.483245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.484011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.484363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.484713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.496875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:51.498463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-60665dff-5c33-44db-8aa8-814eb8e8bc87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:45:51.514042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:45:51.515078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:45:51.516082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:45:51.516082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:45:51.516082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:45:51.518205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 16/30] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:45:51.519637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56feb83-273f-451e-bb5a-69a63b9ddc2b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.520053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56feb83-273f-451e-bb5a-69a63b9ddc2b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.520475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56feb83-273f-451e-bb5a-69a63b9ddc2b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.522134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 17/31] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:51.523369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56feb83-273f-451e-bb5a-69a63b9ddc2b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.535357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 15/32] 10.222.0.61 () {60 vars in 1131 bytes} [Sat Sep 24 14:45:51 2022] POST /identity/v3/roles => generated 219 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:51.536561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.563302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:51.631508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:51.671026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56feb83-273f-451e-bb5a-69a63b9ddc2b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.689167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e508ccd-17d3-4474-b2e5-1ea3133a2f02 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.710763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a823500-35e8-4bee-8ec1-1cd716ba8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.711664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823500-35e8-4bee-8ec1-1cd716ba8058 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.712117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823500-35e8-4bee-8ec1-1cd716ba8058 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.712537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823500-35e8-4bee-8ec1-1cd716ba8058 None admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.714384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a823500-35e8-4bee-8ec1-1cd716ba8058 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:51.716049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a823500-35e8-4bee-8ec1-1cd716ba8058 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:51.718283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 18/33] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:45:51 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:51.719891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320586f1-146f-4db5-a94c-05269be37eda None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.720378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320586f1-146f-4db5-a94c-05269be37eda None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.720850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320586f1-146f-4db5-a94c-05269be37eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.737041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.737975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.738268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.738548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.739225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:51.739522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:51.741082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd77538-589f-4b4d-9ecb-66194e4e876a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:51.746804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-320586f1-146f-4db5-a94c-05269be37eda None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:51.748044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-320586f1-146f-4db5-a94c-05269be37eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:51.762993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 16/34] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:51.764362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d8030-9a42-4e0d-b18e-366c5b38e75b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.764658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d8030-9a42-4e0d-b18e-366c5b38e75b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.764936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d8030-9a42-4e0d-b18e-366c5b38e75b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.770926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 17/35] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:51.771528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d8030-9a42-4e0d-b18e-366c5b38e75b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.838944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 19/36] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:45:51.839881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320586f1-146f-4db5-a94c-05269be37eda None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:51.863491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:51.904124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d8030-9a42-4e0d-b18e-366c5b38e75b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.948230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:51.964213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1e6d1-cff9-4300-8069-6ca88a39f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:51.965662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e6d1-cff9-4300-8069-6ca88a39f328 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.966484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e6d1-cff9-4300-8069-6ca88a39f328 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.967001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e6d1-cff9-4300-8069-6ca88a39f328 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.968377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320586f1-146f-4db5-a94c-05269be37eda None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:51.968639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1e6d1-cff9-4300-8069-6ca88a39f328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:51.970520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1e6d1-cff9-4300-8069-6ca88a39f328 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:51.973142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 18/37] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:51.974636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:51.975128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:51.975607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:51.998347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:52.000658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:52.014245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:52.014870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:52.015152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:52.015429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:52.016395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:52.018222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:52.090306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 19/38] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:45:52.091642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:52.190204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 20/39] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:45:51 2022] POST /identity/v3/projects => generated 301 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:52.191664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:52.233105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c90cb-46b9-4a3e-9d3a-8a7d9887a11d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:52.262266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:52.279152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274fa8-3021-4534-b2c9-04f2f66f0c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:52.279914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274fa8-3021-4534-b2c9-04f2f66f0c8d None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:52.280208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274fa8-3021-4534-b2c9-04f2f66f0c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:52.280496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274fa8-3021-4534-b2c9-04f2f66f0c8d None admin] PATH_INFO: `/v3/domains/default/users/3ea61b3146144581bb15be72dcc836fe/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:52.283220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:52.284625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274fa8-3021-4534-b2c9-04f2f66f0c8d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3ea61b3146144581bb15be72dcc836fe, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:52.286747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274fa8-3021-4534-b2c9-04f2f66f0c8d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:52.308143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 20/40] 10.222.0.61 () {58 vars in 1361 bytes} [Sat Sep 24 14:45:52 2022] PUT /identity/v3/domains/default/users/3ea61b3146144581bb15be72dcc836fe/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:52.314994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a134-488e-48ca-b070-dca3c1c19c6a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:52.315327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a134-488e-48ca-b070-dca3c1c19c6a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:52.315560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a134-488e-48ca-b070-dca3c1c19c6a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:52.317391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 21/41] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:52.324003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:52.324428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:52.324716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:52.330919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5f11-b7ab-47d0-a45b-735e5ccd0ed1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:52.357966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:52.359223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:52.386990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:52.387877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:52.388235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:52.388586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:52.389374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:52.391080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:52.411907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 21/42] 10.222.0.61 () {60 vars in 1131 bytes} [Sat Sep 24 14:45:52 2022] POST /identity/v3/roles => generated 225 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:52.418760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:52.485724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 22/43] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:45:52.568515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6ef21-39d2-413c-be39-74248f88e0c4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:52.601158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:52.609775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:52.611425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:52.611716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:52.612002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:52.612961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:52.614507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:52.633085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 22/44] 10.222.0.61 () {60 vars in 1131 bytes} [Sat Sep 24 14:45:52 2022] POST /identity/v3/roles => generated 223 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:45:52.776786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:52.880971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1eeac-6efd-4e35-a1d9-938d445ac71a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:52.999416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:53.022575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acedc63-3551-4729-8903-a5d80a9333dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:53.023745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acedc63-3551-4729-8903-a5d80a9333dc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:53.024028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acedc63-3551-4729-8903-a5d80a9333dc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:53.024300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acedc63-3551-4729-8903-a5d80a9333dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:53.025406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acedc63-3551-4729-8903-a5d80a9333dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:53.026939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acedc63-3551-4729-8903-a5d80a9333dc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:53.028448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 23/45] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:53.034124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:53.131330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865dd05-09fc-4855-9cc9-906e9638c988 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:53.165372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46726ad-c95f-42b5-b48b-d6e9ea109a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:53.166013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46726ad-c95f-42b5-b48b-d6e9ea109a9a None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:53.166305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46726ad-c95f-42b5-b48b-d6e9ea109a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:53.166585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46726ad-c95f-42b5-b48b-d6e9ea109a9a None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:53.167253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46726ad-c95f-42b5-b48b-d6e9ea109a9a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:53.168720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46726ad-c95f-42b5-b48b-d6e9ea109a9a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:53.195026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 23/46] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:45:53 2022] POST /identity/v3/projects => generated 312 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:45:54.581675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5433ed-e0d3-4edd-98b3-1a974c4132b0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.582256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5433ed-e0d3-4edd-98b3-1a974c4132b0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.582360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5433ed-e0d3-4edd-98b3-1a974c4132b0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.583640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 24/47] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:54.589181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.589559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.589858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.608356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:54.609856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:54.647615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:54.678956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 24/48] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:45:54.946476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5433ed-e0d3-4edd-98b3-1a974c4132b0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.047219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5433ed-e0d3-4edd-98b3-1a974c4132b0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.079091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.079732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.080015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.080293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.091202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:45:55.096105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:45:55.097042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:45:55.098098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 25/49] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:45:55.103594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.199222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f1eca-c647-48e0-b7a8-dc4c69f2f6ff None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.234628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.235264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.235548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.235826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.236521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:55.236833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.238584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05105b36-0eb9-430b-bf1b-0e0e6d7bd0e1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.248971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 25/50] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:55.253755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.295003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:55.339697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f0d8e-e6ba-4356-b6dd-1a5e1cc1fbe9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.339989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f0d8e-e6ba-4356-b6dd-1a5e1cc1fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.340273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f0d8e-e6ba-4356-b6dd-1a5e1cc1fbe9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.341619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 26/51] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:45:55.346488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.346781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.347065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.352731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8030274-f53e-4cfc-b5f6-560f3383d552 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.388144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.388795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.389077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.389353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.390529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.392023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.393843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 26/52] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:55.399905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.477858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:55.479107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:55.503998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c81e91-29c0-4d37-a985-8151786e78e9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.505642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:55.513213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:55.538565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.539194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.539480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.539766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.540887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.542406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.543972 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 27/53] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:45:55.547777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 27/54] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:45:55.549670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.647881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5d34-b584-4578-b2d1-e83adaec8b1d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.679766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.680368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.680653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.680927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.681646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.domain.id=default, user.id=3ea61b3146144581bb15be72dcc836fe) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:55.681991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.683443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.688938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 28/55] 10.222.0.61 () {56 vars in 1322 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/role_assignments?user.id=3ea61b3146144581bb15be72dcc836fe&scope.domain.id=default&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 530 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:55.814631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.913240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d908fc-edcf-4290-bd22-33223c87c5f8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.947465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede27f9c-467a-44d3-82aa-43affa8023f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.948104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede27f9c-467a-44d3-82aa-43affa8023f0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.948393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede27f9c-467a-44d3-82aa-43affa8023f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.948677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede27f9c-467a-44d3-82aa-43affa8023f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.949777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede27f9c-467a-44d3-82aa-43affa8023f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.951275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede27f9c-467a-44d3-82aa-43affa8023f0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.952848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 28/56] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:55.958440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.999285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:56.055847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e827f8e-4da3-4354-bb87-2802709b1ff0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:56.088474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0749f-6fca-4a81-8399-0a8f689e1419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:56.089025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0749f-6fca-4a81-8399-0a8f689e1419 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:56.089312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0749f-6fca-4a81-8399-0a8f689e1419 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:56.089592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0749f-6fca-4a81-8399-0a8f689e1419 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:56.090276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0749f-6fca-4a81-8399-0a8f689e1419 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:56.091733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0749f-6fca-4a81-8399-0a8f689e1419 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:56.119408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 29/57] 10.222.0.61 () {60 vars in 1140 bytes} [Sat Sep 24 14:45:55 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:58.232244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499e5f-e1cb-4989-831b-52eb69d79d77 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:58.232731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499e5f-e1cb-4989-831b-52eb69d79d77 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:58.232815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499e5f-e1cb-4989-831b-52eb69d79d77 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:58.234192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 29/58] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:45:58.239677 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:58.240062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:58.240235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:58.258511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:58.259697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:58.323515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 30/59] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:45:58.596065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499e5f-e1cb-4989-831b-52eb69d79d77 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:58.626820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:58.693978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499e5f-e1cb-4989-831b-52eb69d79d77 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:58.726015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e600c7-bd5f-4c81-81e0-ffaa9742a9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:58.726642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e600c7-bd5f-4c81-81e0-ffaa9742a9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:58.726926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e600c7-bd5f-4c81-81e0-ffaa9742a9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:58.727204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e600c7-bd5f-4c81-81e0-ffaa9742a9d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:58.728304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e600c7-bd5f-4c81-81e0-ffaa9742a9d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:58.729743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e600c7-bd5f-4c81-81e0-ffaa9742a9d8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:58.731314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 30/60] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:45:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:58.736599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:58.760515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:58.833840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c3f7f-5a02-4270-898d-ad398a69f064 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:58.866880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b111d0-4943-4929-bf13-f56107801007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:58.867436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b111d0-4943-4929-bf13-f56107801007 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:58.867721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b111d0-4943-4929-bf13-f56107801007 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:58.867998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b111d0-4943-4929-bf13-f56107801007 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:58.868644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b111d0-4943-4929-bf13-f56107801007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:58.870120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b111d0-4943-4929-bf13-f56107801007 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:58.895452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 31/61] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:45:58 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:02.155883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7070e-a031-45b7-86ee-39649713925b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.156896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7070e-a031-45b7-86ee-39649713925b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.157379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7070e-a031-45b7-86ee-39649713925b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.158951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 31/62] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:02.165112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc40c2-59fd-4e4b-b2f3-0ba26ea660c4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.166037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc40c2-59fd-4e4b-b2f3-0ba26ea660c4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.166561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc40c2-59fd-4e4b-b2f3-0ba26ea660c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.195752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f1dc40c2-59fd-4e4b-b2f3-0ba26ea660c4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:02.197337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f1dc40c2-59fd-4e4b-b2f3-0ba26ea660c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:02.287346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 32/63] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:02.306536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4249c89-87dc-4825-84e1-f45c06389c2f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.307166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4249c89-87dc-4825-84e1-f45c06389c2f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.307646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4249c89-87dc-4825-84e1-f45c06389c2f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.309290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 32/64] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:02.315388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.315998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.316490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.354923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:02.356480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:02.358820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:02.415107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db450e38-b90e-4888-9fa1-1963f100711a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.415791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db450e38-b90e-4888-9fa1-1963f100711a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.416270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db450e38-b90e-4888-9fa1-1963f100711a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.417832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 33/65] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:02.423944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.424474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.424974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.439809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 33/66] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:02.446509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:02.447941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:02.518346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 34/67] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:02.602187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:02.727109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c4582-38cc-40f4-9f85-5ba98a87c08d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:02.777562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:02.778235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.778520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.778795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.787072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:02.794768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:02.799827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:02.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:02.801429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 34/68] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:02.807741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:02.835022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:02.839625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:02.917666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc0dc7-d02f-44c7-bc98-e8cbb6e1c0f4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:02.957929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b92a0-9a6c-4320-93c5-289507f0e100 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:02.960917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:02.961610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:02.961932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:02.962208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:02.962882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:02.963194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:02.964570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:02.974011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 35/69] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:02.974893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.000201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bcf06-727f-4c37-bb35-bd0620c6f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.000976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bcf06-727f-4c37-bb35-bd0620c6f978 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.001083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bcf06-727f-4c37-bb35-bd0620c6f978 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.001300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bcf06-727f-4c37-bb35-bd0620c6f978 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.011945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bcf06-727f-4c37-bb35-bd0620c6f978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-b46bcf06-727f-4c37-bb35-bd0620c6f978 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:03.016737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:03.017568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:03.018388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 35/70] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:03.019030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd77c-3fa0-4126-b35d-d70f7f1973bf None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.020065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd77c-3fa0-4126-b35d-d70f7f1973bf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.020065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfd77c-3fa0-4126-b35d-d70f7f1973bf None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.020720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 36/71] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:03.026086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfd77c-3fa0-4126-b35d-d70f7f1973bf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.074008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.083350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04061-caa2-47fe-8d16-a44b4d696434 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.086408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.119777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.120414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.120704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.120984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.133210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:03.134294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfd77c-3fa0-4126-b35d-d70f7f1973bf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:03.139269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:03.140103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:03.140873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 36/72] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:03.141512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.169870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c71771-9087-4bf0-bdd9-0a2549b8d6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.170532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c71771-9087-4bf0-bdd9-0a2549b8d6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.170804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c71771-9087-4bf0-bdd9-0a2549b8d6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.171080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c71771-9087-4bf0-bdd9-0a2549b8d6b4 None admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.172209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c71771-9087-4bf0-bdd9-0a2549b8d6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.173674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c71771-9087-4bf0-bdd9-0a2549b8d6b4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.179752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 37/73] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.179752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af702b-2bcd-48f5-b5cf-f103c978963e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.179752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af702b-2bcd-48f5-b5cf-f103c978963e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.179752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af702b-2bcd-48f5-b5cf-f103c978963e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.194007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d2af702b-2bcd-48f5-b5cf-f103c978963e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:03.195209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d2af702b-2bcd-48f5-b5cf-f103c978963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:03.248669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2e8b7-163b-48f6-b122-84272c9d1bd7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.273956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 38/74] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:03.273956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ff4c-296a-48c0-ba99-6eeb4ca9a50a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.273956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ff4c-296a-48c0-ba99-6eeb4ca9a50a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.273956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ff4c-296a-48c0-ba99-6eeb4ca9a50a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.273956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 39/75] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:03.273956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573ff4c-296a-48c0-ba99-6eeb4ca9a50a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.284289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.285032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.285483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.285951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.286752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:03.287096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.289144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.297179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 37/76] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.297179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.316844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.338706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.386383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573ff4c-296a-48c0-ba99-6eeb4ca9a50a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.404037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62af59d-0957-44fa-b445-1b2194d81651 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.420339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.421142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.421602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.422066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.422739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:03.423052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.424456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83502f53-cec3-4d45-aee5-884ccf37f4ae None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.437970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 40/77] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.437970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.437970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.437970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.445321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.445965 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.446257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.446534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.457548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:03.458786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:03.523163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.524948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.528728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 41/78] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:03.532008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.546068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.588663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.638491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 38/79] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:03.639159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.645998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b282e-358b-41b0-84d0-3b7d0b8005e3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.684259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.685240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.685526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.686131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.699364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.700839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.702655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 42/80] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.703257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.754955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486be3-6bb7-4cee-b6c1-d3ddf4e445b4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.770378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.793583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a58db-8fae-434c-8038-875647520d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.794394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a58db-8fae-434c-8038-875647520d6d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.794525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a58db-8fae-434c-8038-875647520d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.794762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a58db-8fae-434c-8038-875647520d6d None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.796085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a58db-8fae-434c-8038-875647520d6d None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.797375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a58db-8fae-434c-8038-875647520d6d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.799203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 39/81] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.802998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a58db-8fae-434c-8038-875647520d6d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.823295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac96886-90a2-404d-bb63-18c5ccb32e39 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.830921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:03.858614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.859244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.859526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.859809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.860911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.862676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.864235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 43/82] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.866912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.907334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a58db-8fae-434c-8038-875647520d6d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.941210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:03.941861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:03.942208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:03.942513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:03.943953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:03.945465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:03.947160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 40/83] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:03.947938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:03.970829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09876a-c488-48b2-ac5c-35ba07151b35 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:03.999031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.004601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.005239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.005522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.005799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.047378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244084-06a7-400e-a657-ca35dabb8636 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.047962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.072563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.074060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.080074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.080704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.080990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.081273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.091733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:04.096612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:04.097461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:04.098225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 41/84] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:04.098825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.180831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 44/85] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:04.181650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.202261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbe668-ef2d-41f7-a453-44d07fc5048b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.213133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.237047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.237929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.238286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.238566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.255034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.281098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53adc7e-f97c-430d-89e8-6acc1f6d8b86 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.304422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.305840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.314469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.315080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.315362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.315637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.316280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.317720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.339386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 45/86] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:04 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:04.340207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.427342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 42/87] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:04.432537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.441223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.463563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712967-f6d6-48f6-83cf-108fb1c875a4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.496562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.504947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.505555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.506126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.506653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.507931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=3ea61b3146144581bb15be72dcc836fe) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:04.508240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.509699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.516077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 46/88] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/role_assignments?user.id=3ea61b3146144581bb15be72dcc836fe&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:04.516512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.558532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e0f22-7bf5-4572-ba61-b871358f00c8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.611524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.612984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.613423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.613853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.614962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:04.615531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.617679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.632490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 43/89] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:04.638314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.662255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa0832-6bbf-4ae1-bc2a-209663bf78e6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.711313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.712211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.712646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.713060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.714399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.716015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.717821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 47/90] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:04.722904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.728570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.774713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73d502-f8c5-4829-977b-ed95f4508ced None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.819159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:04.829754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.830993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.831480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.832524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.834169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.836854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.838890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 44/91] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:04.839982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.868742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d6c51-7f2a-47c2-9f96-f7dbe965d1b8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:04.904296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:04.904944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:04.905233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:04.905821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:04.906696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:04.908166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:04.909955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 48/92] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:04.910563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:04.974770 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346dfa-ef1d-4a01-ae0b-67954d38e7a8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:05.017061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:05.017681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:05.018008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:05.018292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:05.019031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] RBAC: Adding query filter params (role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=2ec8a60b86c745078741d42a19dcb7a8) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:05.019349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:05.020799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:05.029536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 45/93] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/role_assignments?user.id=2ec8a60b86c745078741d42a19dcb7a8&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:05.029681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad8fb7-4f6b-4514-9716-ed734dc51cde None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:05.030321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:05.042188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:05.095928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:05.102388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:05.103193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:05.103615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:05.105069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:05.106016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=2ec8a60b86c745078741d42a19dcb7a8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:05.106447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:05.108074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:05.115830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 49/94] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:04 2022] GET /identity/v3/role_assignments?user.id=2ec8a60b86c745078741d42a19dcb7a8&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:05.215506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ef520-5967-4982-9815-927b4809d515 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:05.256153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:05.256810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:05.257106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:05.257389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:05.333516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:05.335579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:05.338539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:05.445247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 46/95] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:05.450832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:05.568147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5ec7-f38e-4b6b-892d-5bbe774adc4b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:05.618645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a61b0-d49d-4731-a802-970733aef129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:05.619302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a61b0-d49d-4731-a802-970733aef129 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:05.619596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a61b0-d49d-4731-a802-970733aef129 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:05.619880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a61b0-d49d-4731-a802-970733aef129 None admin] PATH_INFO: `/v3/projects/81895b739f904f0ebf8215cd12565b86` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:05.621083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a61b0-d49d-4731-a802-970733aef129 None admin] RBAC: Authorizing `identity:get_project(project_id=81895b739f904f0ebf8215cd12565b86)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:05.622675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a61b0-d49d-4731-a802-970733aef129 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:05.624333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 50/96] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:05 2022] GET /identity/v3/projects/81895b739f904f0ebf8215cd12565b86 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:05.630335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:05.749345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55742457-968b-4971-9280-58b9ef647e93 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:05.765439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:05.789772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:05.790426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:05.790716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:05.790997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:05.791739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=81895b739f904f0ebf8215cd12565b86, user.id=2ec8a60b86c745078741d42a19dcb7a8) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:05.792056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:05.793514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1f94c-3645-4d6c-be41-fb7d22129e67 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:05.799469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 47/97] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:05 2022] GET /identity/v3/role_assignments?user.id=2ec8a60b86c745078741d42a19dcb7a8&scope.project.id=81895b739f904f0ebf8215cd12565b86&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:07.354977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6155-9697-41a0-985f-9a4f315042f9 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.355489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6155-9697-41a0-985f-9a4f315042f9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.355594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6155-9697-41a0-985f-9a4f315042f9 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.357061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 51/98] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:07.362775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d60985-f6c5-43c0-8d8b-3cbb86f012e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.363565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d60985-f6c5-43c0-8d8b-3cbb86f012e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.364042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d60985-f6c5-43c0-8d8b-3cbb86f012e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.389109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-57d60985-f6c5-43c0-8d8b-3cbb86f012e9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:07.390717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-57d60985-f6c5-43c0-8d8b-3cbb86f012e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.474826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 48/99] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:07.559439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa02fa8-1ff3-441d-8c1c-b6ed53d29066 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.560092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa02fa8-1ff3-441d-8c1c-b6ed53d29066 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.560582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa02fa8-1ff3-441d-8c1c-b6ed53d29066 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.562174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 52/100] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:07.577995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb9f8c-749c-4936-add5-429592f24f6c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.578623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb9f8c-749c-4936-add5-429592f24f6c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.579122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb9f8c-749c-4936-add5-429592f24f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.602809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-13cb9f8c-749c-4936-add5-429592f24f6c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:07.604015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-13cb9f8c-749c-4936-add5-429592f24f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.693167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 49/101] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:07.873183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa02fa8-1ff3-441d-8c1c-b6ed53d29066 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.926394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7da49c-0a83-4d9c-aa4c-977640a8fd6a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.927009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7da49c-0a83-4d9c-aa4c-977640a8fd6a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.927498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7da49c-0a83-4d9c-aa4c-977640a8fd6a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.929128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 50/102] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:07.934330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.934620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.934901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.964141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:07.965259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.980970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.011732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.050860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa02fa8-1ff3-441d-8c1c-b6ed53d29066 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.087699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 51/103] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:08.099517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.100156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.100467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.100750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.113079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:08.117942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:08.118766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:08.119897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 53/104] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:08.125021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.188863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.270899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbd08b-b567-426a-8e25-96d2f13d5135 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.333083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.333698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.334022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.334302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.334999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:08.335306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.336740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.352662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 52/105] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:08.353424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3c12-3a48-46b0-8104-653210926894 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.359671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.386184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.389752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.390690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.391171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.391587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.392915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.394584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.396854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 54/106] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:08.402571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.405982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.529059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74d1d5-b6ce-49ba-ae08-ee8a1ef6b586 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.544210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040f90-9deb-43b1-979b-d00397f20773 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.572197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.573080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.573427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.573921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] PATH_INFO: `/v3/users/12d50ae333ae49d39c14149a141c8812` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.585818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1cf76-3a28-4009-8fbe-2961585b8fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.586545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1cf76-3a28-4009-8fbe-2961585b8fb6 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.586899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1cf76-3a28-4009-8fbe-2961585b8fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.587490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1cf76-3a28-4009-8fbe-2961585b8fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.588234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1cf76-3a28-4009-8fbe-2961585b8fb6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.589751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1cf76-3a28-4009-8fbe-2961585b8fb6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.592737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] RBAC: Authorizing `identity:get_user(user_id=12d50ae333ae49d39c14149a141c8812)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.594436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.597056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 53/107] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/users/12d50ae333ae49d39c14149a141c8812 => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:08.603065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.631770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 55/108] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:46:08 2022] POST /identity/v3/projects => generated 302 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:08.633987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a870-12b2-4bc4-9c1c-fd99828e9860 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.634475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a870-12b2-4bc4-9c1c-fd99828e9860 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.634992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a870-12b2-4bc4-9c1c-fd99828e9860 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.636995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 56/109] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:08.638284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a870-12b2-4bc4-9c1c-fd99828e9860 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.702502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.770838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.782700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a870-12b2-4bc4-9c1c-fd99828e9860 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.796449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e36a49-0324-4b23-a354-1991e2bb0354 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.830814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23668fdd-df2d-4227-a6b3-721ad44dc74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.831739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23668fdd-df2d-4227-a6b3-721ad44dc74f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.832319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23668fdd-df2d-4227-a6b3-721ad44dc74f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.832769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23668fdd-df2d-4227-a6b3-721ad44dc74f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.835182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5a31f-1b96-4512-8c1f-cbb724f608d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.835919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5a31f-1b96-4512-8c1f-cbb724f608d7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.836437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5a31f-1b96-4512-8c1f-cbb724f608d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.836560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5a31f-1b96-4512-8c1f-cbb724f608d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.849193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5a31f-1b96-4512-8c1f-cbb724f608d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-92a5a31f-1b96-4512-8c1f-cbb724f608d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:08.855018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:08.855904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:08.856643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 54/110] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:08.862857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71ea0-9bcb-472b-98b9-4bfd26fce252 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.862857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71ea0-9bcb-472b-98b9-4bfd26fce252 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.862857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71ea0-9bcb-472b-98b9-4bfd26fce252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.887521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b6f71ea0-9bcb-472b-98b9-4bfd26fce252 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:08.889007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b6f71ea0-9bcb-472b-98b9-4bfd26fce252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:08.926386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23668fdd-df2d-4227-a6b3-721ad44dc74f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.927898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23668fdd-df2d-4227-a6b3-721ad44dc74f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.974269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.979684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 55/111] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:08.981509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69641d84-da0d-4177-b34f-b57bf6fd19ec None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.982008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69641d84-da0d-4177-b34f-b57bf6fd19ec None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.983103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69641d84-da0d-4177-b34f-b57bf6fd19ec None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.985045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 56/112] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:08.986109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69641d84-da0d-4177-b34f-b57bf6fd19ec None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.060794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 57/113] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:09.066569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.066977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.067356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.086061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:09.087470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:09.160895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 58/114] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:09.162094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.165559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.259463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69641d84-da0d-4177-b34f-b57bf6fd19ec None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.284181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.316586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.317129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.317414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.317694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.318401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:09.318711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.320704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.334667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae604f1c-c6fb-4970-8f6d-4150ff4fe944 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.341889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 57/115] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:09.342825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.383008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a021e-9792-4b8f-9783-9da3560c6768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.383657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a021e-9792-4b8f-9783-9da3560c6768 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.383944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a021e-9792-4b8f-9783-9da3560c6768 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.384221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a021e-9792-4b8f-9783-9da3560c6768 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.385350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a021e-9792-4b8f-9783-9da3560c6768 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.387371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a021e-9792-4b8f-9783-9da3560c6768 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.390089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 59/116] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:09.391480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c7514-b200-4ceb-a649-8d1e86a476fb None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.391769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c7514-b200-4ceb-a649-8d1e86a476fb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.392051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c7514-b200-4ceb-a649-8d1e86a476fb None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.394125 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 60/117] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:09.395286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c7514-b200-4ceb-a649-8d1e86a476fb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.458431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd347b3e-687d-406c-b74a-9adc531d9d8d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.495487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.503331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.504056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.504864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.505216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.524600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:09.529517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:09.530909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:09.531920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 58/118] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:09.536135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.542517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c7514-b200-4ceb-a649-8d1e86a476fb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.582109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19343cf8-b114-4551-a192-fd7b551ed583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.582678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19343cf8-b114-4551-a192-fd7b551ed583 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.582962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19343cf8-b114-4551-a192-fd7b551ed583 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.583241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19343cf8-b114-4551-a192-fd7b551ed583 None admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.584591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19343cf8-b114-4551-a192-fd7b551ed583 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.585824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19343cf8-b114-4551-a192-fd7b551ed583 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.587602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 61/119] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:09.588728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.589018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.589291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.608910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:09.610124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:09.612209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.650104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26fa69-9fd0-427e-8200-d50a8b1c62a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.685513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 62/120] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:09.685513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.696729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.697548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.698091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.698512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.699543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] RBAC: Adding query filter params (role.id=2a56bd70614445349f9d810091b9f046, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=12d50ae333ae49d39c14149a141c8812) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:09.699861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.701640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.717972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 59/121] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/role_assignments?user.id=12d50ae333ae49d39c14149a141c8812&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=2a56bd70614445349f9d810091b9f046 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:09.717972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.841443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1364c-a7e1-420e-b31d-389d238487c5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.859663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.883402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.885250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.885250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.885250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.885250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:09.885521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.887689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.892809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.901779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 63/122] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:09.903435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.973464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ba270-65cf-4e02-83f2-27cc5f54f6e3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.024461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.025445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.025948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.026414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.037104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:10.043244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:10.044099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:10.044946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 60/123] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:10.045557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.070238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dabb0-d672-42bd-af8f-747f1493590a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.121459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.122446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.122989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.123277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.124430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.126198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.129268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 64/124] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:10.130134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.167992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35170bfd-6a23-4065-87f9-bfe21d9c3f06 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.184150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.208377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.209195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.209547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.209931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.211343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.212860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.219309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 61/125] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:10.219309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.231537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.275356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada4057-8f3b-45ab-9b79-6a8ab32fc603 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.317948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.319332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.319794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.320520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.331329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5051e1-771c-4b38-90d4-e20511134c3c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.334347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:10.341033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:10.341874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 65/126] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:10.343294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.378181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.379260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.380454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.380953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.381848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:10.382392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.383964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.394606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 62/127] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:10.395785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.451462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.458231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4b7a-d585-42f3-ae62-5bf833c1dd82 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.489518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.499843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.500553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.500834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.501416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/3ea61b3146144581bb15be72dcc836fe/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.503467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=16631e93412a46bf9ab3e96da45f3ed6, user_id=3ea61b3146144581bb15be72dcc836fe, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.504934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.521272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 66/128] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:10 2022] PUT /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/3ea61b3146144581bb15be72dcc836fe/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:10.526735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.533138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f02f2e-fbb7-4f22-a78b-e8cb1fc1b878 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.619604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.619604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.619604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.619604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.619999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.619999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.619999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 63/129] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:10.620697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.732682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ccea9-3c6e-4fed-a5d2-1aed280e12b0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.782127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.782754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.783037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.783316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.783986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:10.784296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.785685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.792104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca50aa0-7d88-4f2b-b0e2-7c697775bb96 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.794948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 67/130] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:10.795785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.808423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.839302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.839937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.840223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.840502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.843083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.844986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.847304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 64/131] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:10.848336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.859111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.915155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10a358-105f-456d-a0d2-fd3f377e248e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.956171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.956994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.957285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.957945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/2ec8a60b86c745078741d42a19dcb7a8/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.960375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] RBAC: Authorizing `identity:create_grant(project_id=16631e93412a46bf9ab3e96da45f3ed6, user_id=2ec8a60b86c745078741d42a19dcb7a8, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.962527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.967447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d2c64-3ee7-4eb7-b145-e5300f6ea972 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.993055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 68/132] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:10 2022] PUT /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/2ec8a60b86c745078741d42a19dcb7a8/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:10.994336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.054526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.055563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.057701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.058208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.060510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.063195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.066736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 65/133] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:11.072979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.187373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:11.234170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad800ca-ae27-4385-af66-38636eefff76 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.247231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173da0a-e2df-4ce2-a252-e4a246f04616 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.270523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:11.275593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce3b3a-9077-4eba-840f-881d598f24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.276504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce3b3a-9077-4eba-840f-881d598f24e6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.277301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce3b3a-9077-4eba-840f-881d598f24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.277819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce3b3a-9077-4eba-840f-881d598f24e6 None admin] PATH_INFO: `/v3/projects/81895b739f904f0ebf8215cd12565b86` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.279636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce3b3a-9077-4eba-840f-881d598f24e6 None admin] RBAC: Authorizing `identity:get_project(project_id=81895b739f904f0ebf8215cd12565b86)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.281476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce3b3a-9077-4eba-840f-881d598f24e6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.283858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 66/134] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:11 2022] GET /identity/v3/projects/81895b739f904f0ebf8215cd12565b86 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:11.285205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e1ab0-a241-4ba2-9a7c-dcf163b1d0a6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.285824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e1ab0-a241-4ba2-9a7c-dcf163b1d0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.286252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68636ce1-7520-42a5-9e34-8793ab443add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.286547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e1ab0-a241-4ba2-9a7c-dcf163b1d0a6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.286903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68636ce1-7520-42a5-9e34-8793ab443add None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.287193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68636ce1-7520-42a5-9e34-8793ab443add None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.287470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68636ce1-7520-42a5-9e34-8793ab443add None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.288951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68636ce1-7520-42a5-9e34-8793ab443add None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.289165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 67/135] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:11.290167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e1ab0-a241-4ba2-9a7c-dcf163b1d0a6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.290470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68636ce1-7520-42a5-9e34-8793ab443add None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.292223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 69/136] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:11.301809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.302256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.302653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.325234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:11.327555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:11.410656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 70/137] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:11.412059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.452238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e1ab0-a241-4ba2-9a7c-dcf163b1d0a6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.478830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:11.500661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.501530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.502030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.502468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] PATH_INFO: `/v3/projects/81895b739f904f0ebf8215cd12565b86/users/2ec8a60b86c745078741d42a19dcb7a8/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.504628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] RBAC: Authorizing `identity:create_grant(project_id=81895b739f904f0ebf8215cd12565b86, user_id=2ec8a60b86c745078741d42a19dcb7a8, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.506343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.527232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 68/138] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:11 2022] PUT /identity/v3/projects/81895b739f904f0ebf8215cd12565b86/users/2ec8a60b86c745078741d42a19dcb7a8/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:11.537942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b012934-78c9-44e9-b27b-741bb2fa717a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.547899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:11.589152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.590299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.590819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.591328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/2ec8a60b86c745078741d42a19dcb7a8/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.594776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] RBAC: Authorizing `identity:create_grant(project_id=16631e93412a46bf9ab3e96da45f3ed6, user_id=2ec8a60b86c745078741d42a19dcb7a8, role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.596648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.615848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 71/139] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:11 2022] PUT /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/2ec8a60b86c745078741d42a19dcb7a8/roles/c374f9c914654fa8aad1636ed6f12185 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:11.935567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:12.046406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbe7dd-a7e9-4a0e-8c13-8cb4256ffefa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:12.085888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36ea93-3951-48ef-9acd-3bbfaadc1206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:12.086881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36ea93-3951-48ef-9acd-3bbfaadc1206 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:12.087355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36ea93-3951-48ef-9acd-3bbfaadc1206 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:12.087817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36ea93-3951-48ef-9acd-3bbfaadc1206 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:12.089163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36ea93-3951-48ef-9acd-3bbfaadc1206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:12.090836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36ea93-3951-48ef-9acd-3bbfaadc1206 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:12.092989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 69/140] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:12.098327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:12.232064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9838a6a-8d2a-4d9d-b70f-76ec02c18b8d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:12.281452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30693c66-94fb-4c11-a3c6-76f6ff573e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:12.282107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30693c66-94fb-4c11-a3c6-76f6ff573e28 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:12.282391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30693c66-94fb-4c11-a3c6-76f6ff573e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:12.282665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30693c66-94fb-4c11-a3c6-76f6ff573e28 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:12.283344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30693c66-94fb-4c11-a3c6-76f6ff573e28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:12.284829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30693c66-94fb-4c11-a3c6-76f6ff573e28 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:12.314436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 72/141] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:12 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:12.334204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573fff3-9d2e-4edd-a522-0794d328c7db None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:12.334509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573fff3-9d2e-4edd-a522-0794d328c7db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:12.334850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573fff3-9d2e-4edd-a522-0794d328c7db None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:12.336377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 70/142] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:46:12.341890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:12.342217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:12.342480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:12.364012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:12.365738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:12.386956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:12.451236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 73/143] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:12.856056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573fff3-9d2e-4edd-a522-0794d328c7db None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:12.931707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:13.002582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573fff3-9d2e-4edd-a522-0794d328c7db None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:13.048937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:13.049579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.049868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.050167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.068430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:13.073890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:13.074731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:13.075476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:13.076487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 71/144] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:13.081175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:13.208150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32feca-c805-44ec-bbe1-17f145ed70cc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:13.246463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:13.247083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.247361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.247633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.248299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:13.248605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:13.250019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:13.262078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 74/145] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:13.264759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:13.376315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd69d9c-d6d2-45a3-831d-b7bae2489155 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:13.378689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:13.412598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:13.417534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.417835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.418157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] PATH_INFO: `/v3/users/12d50ae333ae49d39c14149a141c8812` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.421312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] RBAC: Authorizing `identity:get_user(user_id=12d50ae333ae49d39c14149a141c8812)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:13.423703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:13.425403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 72/146] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:13 2022] GET /identity/v3/users/12d50ae333ae49d39c14149a141c8812 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:13.431886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:13.457311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:13.567742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5fcd4-6f28-4a07-a351-e02cfbbc8c81 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:13.609488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8cb7d-a88d-4cd7-932c-4c5a385d0ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:13.610174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8cb7d-a88d-4cd7-932c-4c5a385d0ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.610454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8cb7d-a88d-4cd7-932c-4c5a385d0ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.610725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8cb7d-a88d-4cd7-932c-4c5a385d0ea4 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.611918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8cb7d-a88d-4cd7-932c-4c5a385d0ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:13.613702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8cb7d-a88d-4cd7-932c-4c5a385d0ea4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:13.615533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 75/147] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:13 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:13.621799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:13.744656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0ef9b-ab21-457a-9fff-60a072d78752 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:13.779469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59167bb5-0692-4316-bf7d-c9098ec4a65a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.779776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59167bb5-0692-4316-bf7d-c9098ec4a65a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.780027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59167bb5-0692-4316-bf7d-c9098ec4a65a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.781372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 76/148] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:13.786957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6731-ef4c-4f9e-b877-70016bea5c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:13.787707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6731-ef4c-4f9e-b877-70016bea5c86 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.788534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6731-ef4c-4f9e-b877-70016bea5c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.788903 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6731-ef4c-4f9e-b877-70016bea5c86 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/12d50ae333ae49d39c14149a141c8812/roles/2a56bd70614445349f9d810091b9f046` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.790891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a5442-78c5-4ca4-aa73-73f1b6e53423 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:13.791168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a5442-78c5-4ca4-aa73-73f1b6e53423 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:13.791444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a5442-78c5-4ca4-aa73-73f1b6e53423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:13.805310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d6731-ef4c-4f9e-b877-70016bea5c86 None admin] RBAC: Authorizing `identity:create_grant(project_id=16631e93412a46bf9ab3e96da45f3ed6, user_id=12d50ae333ae49d39c14149a141c8812, role_id=2a56bd70614445349f9d810091b9f046)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:13.806873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d6731-ef4c-4f9e-b877-70016bea5c86 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:13.821729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a23a5442-78c5-4ca4-aa73-73f1b6e53423 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:13.825503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a23a5442-78c5-4ca4-aa73-73f1b6e53423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:13.836078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 73/149] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:13 2022] PUT /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/users/12d50ae333ae49d39c14149a141c8812/roles/2a56bd70614445349f9d810091b9f046 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:13.977479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 77/150] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:14.190095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a3e0-3372-42f4-98bf-0a36e9d5520e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:14.190580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a3e0-3372-42f4-98bf-0a36e9d5520e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:14.190580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a3e0-3372-42f4-98bf-0a36e9d5520e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:14.192056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 74/151] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:14.204026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:14.204524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:14.204604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:14.228748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:14.230006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:14.240982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:14.305983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 78/152] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:14.657133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699a3e0-3372-42f4-98bf-0a36e9d5520e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:14.695046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:14.784397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699a3e0-3372-42f4-98bf-0a36e9d5520e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:14.834264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:14.834943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:14.835227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:14.835499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:14.849735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:14.859431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:14.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:14.861216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 75/153] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:14.867411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:14.953666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:15.020200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86bc03-7ee2-4dc2-b318-25c3a0ee3736 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:15.081621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:15.082265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.082542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.082814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.083481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:15.083784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:15.090733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:15.107670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 79/154] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:15.112055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:15.127170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:15.129068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:15.142043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127413ca-c3b4-408c-a5bf-2728513ffd19 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:15.196523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:15.197133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.197371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.198780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.221282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:15.229163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:15.230056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:15.230985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:15.240058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 76/155] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:15.250280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:15.287373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ea5a2-50b5-4067-a04b-86bac3a1b273 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:15.328501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:15.329198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.329641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.330101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] PATH_INFO: `/v3/users/3ea61b3146144581bb15be72dcc836fe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.331448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea61b3146144581bb15be72dcc836fe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:15.333120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:15.335570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 80/156] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:15 2022] GET /identity/v3/users/3ea61b3146144581bb15be72dcc836fe => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:15.341064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:15.459142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:15.478756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a8b51-c2e5-4bfa-94ea-e9f2ecb18f12 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:15.503486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3318a14-1b59-4968-9750-5416d4946771 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:15.534883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:15.535490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.535773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.536048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.537004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:15.537004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:15.539074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:15.556583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339579-015a-4e5a-a257-dc06c5c8c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:15.557713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339579-015a-4e5a-a257-dc06c5c8c92f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.558731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 77/157] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:15.560608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339579-015a-4e5a-a257-dc06c5c8c92f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.563595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339579-015a-4e5a-a257-dc06c5c8c92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.590170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:15.638379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:15.676902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b339579-015a-4e5a-a257-dc06c5c8c92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:15.679338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b339579-015a-4e5a-a257-dc06c5c8c92f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:15.813244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:15.860107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858c11d-2a97-4428-91e7-09f3d2ce8ca0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:15.871441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 81/158] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:15.873029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be4ee8-39d4-42fa-b422-00cd997fbaec None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.873479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be4ee8-39d4-42fa-b422-00cd997fbaec None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.874108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be4ee8-39d4-42fa-b422-00cd997fbaec None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.877201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 82/159] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:15.883060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.883518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.883951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.905938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:15.907415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:15.961611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:15.962292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:15.962574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:15.962848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:15.963991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:15.965444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:15.967359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 78/160] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:15 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:15.968157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:16.018709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 83/161] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:16.019969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:16.167848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bd65f-fca9-470c-8e4b-e5b3a78385b4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:16.178283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:16.185183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:16.208216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043f3bc-d782-4830-8d91-2db27d9eb34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:16.209519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043f3bc-d782-4830-8d91-2db27d9eb34b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.210786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043f3bc-d782-4830-8d91-2db27d9eb34b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.211248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043f3bc-d782-4830-8d91-2db27d9eb34b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.214563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66c77b-6e14-47dd-8b66-1c7eaf123cf0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:16.272060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69660727-bdab-44b5-b84e-5a5aec3e0c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:16.273210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69660727-bdab-44b5-b84e-5a5aec3e0c04 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.273487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69660727-bdab-44b5-b84e-5a5aec3e0c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.273765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69660727-bdab-44b5-b84e-5a5aec3e0c04 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.276372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69660727-bdab-44b5-b84e-5a5aec3e0c04 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:16.277840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69660727-bdab-44b5-b84e-5a5aec3e0c04 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:16.279486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 79/162] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:15 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:16.280587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666cab-3a2c-4c70-a80d-7a47696717c0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.280871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666cab-3a2c-4c70-a80d-7a47696717c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.281150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666cab-3a2c-4c70-a80d-7a47696717c0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.282466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 80/163] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:16.287928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666cab-3a2c-4c70-a80d-7a47696717c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:16.293618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043f3bc-d782-4830-8d91-2db27d9eb34b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:16.295832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043f3bc-d782-4830-8d91-2db27d9eb34b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:16.435662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 84/164] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:16.440074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.440555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.441049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.454562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666cab-3a2c-4c70-a80d-7a47696717c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:16.462221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:16.479358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:16.480544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:16.519354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:16.519956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.520232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.520500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.521224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=3ea61b3146144581bb15be72dcc836fe) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:16.521523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:16.523638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:16.533711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 81/165] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:16 2022] GET /identity/v3/role_assignments?user.id=3ea61b3146144581bb15be72dcc836fe&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:16.538880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:16.582762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:16.634190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 85/166] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:16.701890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c6fff-9471-4837-8942-c7f254706930 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:16.749379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:16.750050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.750335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.750611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.751785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:16.753264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:16.755319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 82/167] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:16 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:16.761330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:16.788484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:16.882781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:16.907954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212af6c-fbb6-4b6b-9a74-124bdafa625e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:16.946912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1540-782a-496a-838f-e1946673c4f8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:16.990353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:16.991364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:16.991913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:16.992382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:16.993329 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=2ec8a60b86c745078741d42a19dcb7a8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:16.993845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:16.995621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:16.997675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:17.012208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 86/168] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:16 2022] GET /identity/v3/role_assignments?user.id=2ec8a60b86c745078741d42a19dcb7a8&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:17.020113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.020767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.021053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.021365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.046502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:17.051836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:17.052707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:17.054194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 83/169] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:17.059029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.154595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.213847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f6aea-6f29-4701-9c4f-a8f948db059e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.269862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09504853-e0a2-4cb0-a576-9c06135f47f3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.292186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.292811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.293089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.293356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.294049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:17.294565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:17.295734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:17.307951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.308599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.308893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.309169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.324289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 87/170] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:17.330109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:17.340895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:17.348047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:17.349019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:17.351192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 84/171] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:17.355553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.366282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:17.458281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:17.485211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612a9df-2a5c-44af-91dd-86570d9d91cd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.527195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.527742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.528025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.528302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.528984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:17.529290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:17.530712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:17.539703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 85/172] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:17.545266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.551730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca9abe-6444-45bb-b6b2-6f5bd13f92c1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.611183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:17.648682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.649329 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.649606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.649870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.651029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:17.651790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c725-7b3a-4974-be7c-003aebf4c2d1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.652498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:17.654417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 88/173] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:17.674955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.691676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.692314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.692598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.692878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] PATH_INFO: `/v3/users/2ec8a60b86c745078741d42a19dcb7a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.694367 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] RBAC: Authorizing `identity:get_user(user_id=2ec8a60b86c745078741d42a19dcb7a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:17.695945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:17.698001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 86/174] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/users/2ec8a60b86c745078741d42a19dcb7a8 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:17.703647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:17.811471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69befdaa-343c-4276-8155-8c55bdda8e49 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.841555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:17.848674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.849304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.849592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.849873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:17.868445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cd0f4-568b-4a76-a776-0fd0437af72e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:17.922964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:17.925650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:17.927632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:17.955193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd44a5-d530-4648-8fff-4eb2f415f8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:17.955823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd44a5-d530-4648-8fff-4eb2f415f8ac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:17.956101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd44a5-d530-4648-8fff-4eb2f415f8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:17.956373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd44a5-d530-4648-8fff-4eb2f415f8ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.107049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd44a5-d530-4648-8fff-4eb2f415f8ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:18.108412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd44a5-d530-4648-8fff-4eb2f415f8ac None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:18.116128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 87/175] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:46:18.121757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:18.179007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:18.271385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4ee4d-25d2-462d-a461-e73c2481a7ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:18.275733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 89/176] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:18.281384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2434b-18d3-4094-ac85-ecb303d89ffd None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.281384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2434b-18d3-4094-ac85-ecb303d89ffd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.281384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2434b-18d3-4094-ac85-ecb303d89ffd None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.283098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 90/177] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:18.284266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89a555-83b3-4bbd-b29a-f2b48eddc901 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.284554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89a555-83b3-4bbd-b29a-f2b48eddc901 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.284838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89a555-83b3-4bbd-b29a-f2b48eddc901 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.286439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 91/178] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:18.287226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89a555-83b3-4bbd-b29a-f2b48eddc901 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:18.318048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca251ed-a7ea-4499-b5e2-bad02e6e1765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:18.319838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca251ed-a7ea-4499-b5e2-bad02e6e1765 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.320695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:18.321590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca251ed-a7ea-4499-b5e2-bad02e6e1765 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.322043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca251ed-a7ea-4499-b5e2-bad02e6e1765 None admin] PATH_INFO: `/v3/projects/81895b739f904f0ebf8215cd12565b86` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.323649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca251ed-a7ea-4499-b5e2-bad02e6e1765 None admin] RBAC: Authorizing `identity:get_project(project_id=81895b739f904f0ebf8215cd12565b86)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:18.325095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca251ed-a7ea-4499-b5e2-bad02e6e1765 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:18.326921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 88/179] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:18 2022] GET /identity/v3/projects/81895b739f904f0ebf8215cd12565b86 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:18.327868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a62dc-e436-4d47-8185-e616d09a8c78 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.328149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a62dc-e436-4d47-8185-e616d09a8c78 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.328422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a62dc-e436-4d47-8185-e616d09a8c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.346684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2e7a62dc-e436-4d47-8185-e616d09a8c78 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:18.347881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2e7a62dc-e436-4d47-8185-e616d09a8c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:18.398858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89a555-83b3-4bbd-b29a-f2b48eddc901 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:18.413747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 89/180] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:18.415133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.415421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.415694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.432996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:18.434193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:18.445009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd905830-62fd-4d2f-a8d3-b32c91feedfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:18.445667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd905830-62fd-4d2f-a8d3-b32c91feedfd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.445997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd905830-62fd-4d2f-a8d3-b32c91feedfd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.446260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd905830-62fd-4d2f-a8d3-b32c91feedfd None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.447488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd905830-62fd-4d2f-a8d3-b32c91feedfd None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:18.449092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd905830-62fd-4d2f-a8d3-b32c91feedfd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:18.451102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 92/181] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:18 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:18.456980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35505b-e16a-43cd-865c-d9fe3ba2930b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.457389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35505b-e16a-43cd-865c-d9fe3ba2930b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.457665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35505b-e16a-43cd-865c-d9fe3ba2930b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.458225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:18.461958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 93/182] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:18.461958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6b558-7e80-47a2-a9f4-ad00621e7281 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.461958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6b558-7e80-47a2-a9f4-ad00621e7281 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.462370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6b558-7e80-47a2-a9f4-ad00621e7281 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.463413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 94/183] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:18.464102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6b558-7e80-47a2-a9f4-ad00621e7281 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:18.512150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 90/184] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:18.513121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:18.590785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:18.599720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6b558-7e80-47a2-a9f4-ad00621e7281 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:18.631962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4b926-1de6-40e6-bc36-d4021aba0c09 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:18.646532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:18.647806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.648260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.648986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.650134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=81895b739f904f0ebf8215cd12565b86, user.id=2ec8a60b86c745078741d42a19dcb7a8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:18.650600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:18.652556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca2044-4a1f-4666-a7ad-5691a105b28d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:18.660999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 95/185] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:18 2022] GET /identity/v3/role_assignments?user.id=2ec8a60b86c745078741d42a19dcb7a8&scope.project.id=81895b739f904f0ebf8215cd12565b86&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:18.663565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.663998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.664825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.669384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:18.670064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.670353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.670632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.671444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] RBAC: Adding query filter params (role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=2ec8a60b86c745078741d42a19dcb7a8) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:18.671766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:18.673287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51d861-d559-4b3b-ad99-6ba9ff03a9b5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:18.680853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 91/186] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:18 2022] GET /identity/v3/role_assignments?user.id=2ec8a60b86c745078741d42a19dcb7a8&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:18.681941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:18.682556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:18.682556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:18.688275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:18.691456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:18.703097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:18.704398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:18.734767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:18.796391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 96/187] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:18.797361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:18.804747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 92/188] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:18.946293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:18.957874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787444c-0749-4b12-99b8-0b7bcbceefd3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:18.967000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.008150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.008150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.008431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.008637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.011026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.012494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.014103 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 97/189] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:19.019344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.110178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1407e7-58d6-460c-af64-7ed1c5693ce4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.151945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.153057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.153586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.154100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.165357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:19.170551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:19.171415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:19.172193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:19.172817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8934f-7d90-48e4-9719-9c4836950d33 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.173502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 93/190] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:19.177649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.194966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.208041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.208655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.208932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.209211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.210457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.211893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.231464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 98/191] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:19 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:46:19.234826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.252121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.293334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852c9dd-5491-4e35-a71f-acae021df088 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.339426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.340075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.340356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.340627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.345583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0495-b037-4462-8fbe-a83e9f382a06 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.353147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:19.358629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:19.359552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:19.360338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 94/192] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:19.360984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.384800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.385432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.385714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.386016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.387245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:19.387563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.389158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.398663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 99/193] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:19.399354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.447965 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.481340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a535b-4465-4d0f-96da-ad946848d6e9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.493547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.522894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a77882-e11c-441b-94ae-8f232c378e1a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.524399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.525047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.525326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.525606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.537982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:19.544386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:19.545257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:19.546054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 95/194] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:19.546716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.562844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.563174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.563436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.563708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.564366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:19.564669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.566077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.575403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 100/195] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:19.576226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.679056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfe8c-b65a-4ebf-a886-029b3437b0ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.707876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64f610-d5c3-4619-90e6-b4fadb72eda5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.724669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.725813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.726319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.726786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] PATH_INFO: `/v3/users/7e8efabafd7e4c729b414f403a1b484b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.729060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.744103 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.744751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.745033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.745325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.746023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:19.746328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.747528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] RBAC: Authorizing `identity:get_user(user_id=7e8efabafd7e4c729b414f403a1b484b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.747717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.750321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.752582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 96/196] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/users/7e8efabafd7e4c729b414f403a1b484b => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:19.753248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.754269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.759533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 101/197] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:19.760364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.872921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623b02f-08be-4b40-a292-a90f1db430f5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.873358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ba004-8dc1-4777-b2b1-c32a2715f229 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:19.909363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.910099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.910526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.910819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.911922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:19.913183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:19.913558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:19.913946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] PATH_INFO: `/v3/users/7e8efabafd7e4c729b414f403a1b484b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:19.915197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] RBAC: Authorizing `identity:get_user(user_id=7e8efabafd7e4c729b414f403a1b484b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.916738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:19.920147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 97/198] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/users/7e8efabafd7e4c729b414f403a1b484b => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:19.920147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:19.991773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.994798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:19.995119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:19.996513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.048838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e39f6-ed94-4a4f-9d44-e59c7af9a772 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.088782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.089949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.090340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.090716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] PATH_INFO: `/v3/users/12d50ae333ae49d39c14149a141c8812` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.092614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] RBAC: Authorizing `identity:get_user(user_id=12d50ae333ae49d39c14149a141c8812)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.094243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.096243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 98/199] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/users/12d50ae333ae49d39c14149a141c8812 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:20.097171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.126034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 102/200] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:20.127514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.216719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c11df-63f1-4b43-a155-ab8f168b119b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.254882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.255639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.258955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4838f3-57a0-4e5a-b2d3-dd3b70147ba5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.267163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.268524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.268951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.269397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.299474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.300123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.300503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.301153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.367711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.369856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.375112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.376486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.476921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.485594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 103/201] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:20.486561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.496604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 99/202] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:20.498077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.530099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.593701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f66b7-350e-4d3c-96db-2593d883ba6c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.611609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9055d-3f8f-464d-ad67-b472f2dd4cec None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.633459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.634216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.634571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.634916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.636678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.638914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.641724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 104/203] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:20.642757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.650680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.651317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.651597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.651869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.652999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.654490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.656131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 100/204] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:20.656958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.728747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.769760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7783b8e-2d69-4dec-83ca-3cda3d55fdde None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.787156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.796602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fef866-ff1b-4497-81aa-112a4fa7efc4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.806852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.807598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.807880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.808151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.809275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.810770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.812467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 105/205] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:20.818459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.840033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26560001-840c-443a-90c7-f74f776f657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.840666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26560001-840c-443a-90c7-f74f776f657a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.840969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26560001-840c-443a-90c7-f74f776f657a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.841727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26560001-840c-443a-90c7-f74f776f657a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.842651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26560001-840c-443a-90c7-f74f776f657a None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=7e8efabafd7e4c729b414f403a1b484b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:20.843081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26560001-840c-443a-90c7-f74f776f657a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.845195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26560001-840c-443a-90c7-f74f776f657a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.853153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 101/206] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/role_assignments?user.id=7e8efabafd7e4c729b414f403a1b484b&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:20.853774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26560001-840c-443a-90c7-f74f776f657a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:20.937371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7679cd-94bf-45f4-ae05-1ddc460ccff9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:20.979047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:20.979977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:20.980297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:20.980588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:20.981812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] RBAC: Adding query filter params (role.id=2a56bd70614445349f9d810091b9f046, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6, user.id=12d50ae333ae49d39c14149a141c8812) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:20.982161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:20.983787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef856f4f-d6f1-463f-be46-4fce68c27093 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:20.987722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:20.993535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 106/207] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/role_assignments?user.id=12d50ae333ae49d39c14149a141c8812&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=2a56bd70614445349f9d810091b9f046 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:21.041249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26560001-840c-443a-90c7-f74f776f657a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:21.088148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:21.089060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:21.089518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:21.089804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:21.090593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] RBAC: Adding query filter params (role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=7e8efabafd7e4c729b414f403a1b484b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:21.090909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:21.092465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d17d5-37c2-4df9-a629-0f67ddcc9ab7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:21.103307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 102/208] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:20 2022] GET /identity/v3/role_assignments?user.id=7e8efabafd7e4c729b414f403a1b484b&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:21.708885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105caf8-f05e-4d4b-a250-fea32488abe1 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:21.709978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105caf8-f05e-4d4b-a250-fea32488abe1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:21.710098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105caf8-f05e-4d4b-a250-fea32488abe1 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:21.711518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 107/209] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:21.717087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:21.717951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:21.718441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:21.737886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:21.739449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:21.741303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca55e7-e56c-4dea-9ad2-12dd5c677c5e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:21.741694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca55e7-e56c-4dea-9ad2-12dd5c677c5e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:21.742436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca55e7-e56c-4dea-9ad2-12dd5c677c5e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:21.745032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 108/210] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:21.751719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:21.752127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:21.752502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:21.768160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:21.779854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:21.781218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:21.822770 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 103/211] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:21.862892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 109/212] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:22.100178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:22.154711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:22.204890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e376-ec2a-4d7c-a7a1-eaa173aceef8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:22.241574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:22.242256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:22.242536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:22.242804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:22.243902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:22.245341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:22.246934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 104/213] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:22.252455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:22.277598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd179d0-6a6f-4556-bbfe-e8e625ccf075 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:22.282342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:22.318539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:22.325372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:22.326173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:22.326450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:22.326718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:22.337625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:22.342788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:22.343664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:22.344441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 110/214] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:22.350980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:22.368398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33eb90c-9496-4a7c-bca0-17f598d0484a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:22.406170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:22.406832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:22.407113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:22.407382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:22.408140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:22.409580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:22.442619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 105/215] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:22 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:46:22.458348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54abf6-455d-470b-9803-79f68640b207 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:22.497152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:22.497788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:22.498092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:22.498369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:22.499056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:22.499361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:22.500805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:22.511417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 111/216] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:22.517261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:22.609139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:22.634478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ae2ff-a2e2-4160-9784-454efbdefebf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:22.672550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:22.673193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:22.673471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:22.673742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] PATH_INFO: `/v3/users/e0f9762c2b744a9eb6c0e7b3032e6e68` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:22.688387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] RBAC: Authorizing `identity:get_user(user_id=e0f9762c2b744a9eb6c0e7b3032e6e68)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:22.689848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:22.691865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 106/217] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:22 2022] GET /identity/v3/users/e0f9762c2b744a9eb6c0e7b3032e6e68 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:22.697819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:22.754091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:22.806289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95493b0b-1512-4bfe-81a4-c9da4e1e3939 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:22.843660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:22.844614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:22.845063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:22.845499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:22.924686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:22.926567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:22.991483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:23.046343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 112/218] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:23.056245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:23.204965 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3433-f438-4af7-8a84-15d224e3de3f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:23.258558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c6b9d-808a-4a19-91fe-528e220fb61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:23.259204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c6b9d-808a-4a19-91fe-528e220fb61a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.259488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c6b9d-808a-4a19-91fe-528e220fb61a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.259814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c6b9d-808a-4a19-91fe-528e220fb61a None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.260985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c6b9d-808a-4a19-91fe-528e220fb61a None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:23.262466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c6b9d-808a-4a19-91fe-528e220fb61a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:23.264123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 107/219] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:23 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:23.272647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:23.333745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff924c03-30d1-4541-9cc7-b39ffab5f687 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.334468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff924c03-30d1-4541-9cc7-b39ffab5f687 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.334940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff924c03-30d1-4541-9cc7-b39ffab5f687 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.336697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 108/220] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:23.341734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d039-3845-4ac3-a5be-9d674c9246a6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.342258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d039-3845-4ac3-a5be-9d674c9246a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.342724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d039-3845-4ac3-a5be-9d674c9246a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.358279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:23.361304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8b27d039-3845-4ac3-a5be-9d674c9246a6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:23.362977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b27d039-3845-4ac3-a5be-9d674c9246a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:23.394231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2455a-f6ad-40dd-b003-f17da5661828 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:23.430690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:23.431454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.431809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.432148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.432879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=e0f9762c2b744a9eb6c0e7b3032e6e68) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:23.433188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:23.434642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 109/221] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:23.434747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423922-55b8-4f43-8871-46b27a613165 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:23.440494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 113/222] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:23 2022] GET /identity/v3/role_assignments?user.id=e0f9762c2b744a9eb6c0e7b3032e6e68&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:23.441829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38eda-c7ad-40b9-8467-6c936082b14e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.442362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38eda-c7ad-40b9-8467-6c936082b14e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.442867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38eda-c7ad-40b9-8467-6c936082b14e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.444807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 110/223] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:23.450678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.450958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.451237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.485955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:23.487584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:23.500463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:23.579156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 114/224] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:23.712613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a38eda-c7ad-40b9-8467-6c936082b14e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:23.819237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a38eda-c7ad-40b9-8467-6c936082b14e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:23.853744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:23.854738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:23.855256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:23.855719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:23.866237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:23.870966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:23.877153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:23.878166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:23.880680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 111/225] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:23.887050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:23.986325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.033647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb9091-188c-4a9f-8e42-ada362974cfd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.103645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddb5da-d7ed-49cf-977e-c3b0a164041a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.109738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.110432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.110709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.110975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.111640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:24.111939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.113590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.138252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 115/226] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:24.142449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.161759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:24.182457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.183137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.183412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.183730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.202702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:24.207754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:24.208633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:24.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 112/227] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:24.218399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.297560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192eac0-0bc3-4ba7-8d3e-1a73ef3ea352 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.305704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:24.345994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.346918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.347290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.347672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] PATH_INFO: `/v3/users/7e8efabafd7e4c729b414f403a1b484b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.348891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] RBAC: Authorizing `identity:get_user(user_id=7e8efabafd7e4c729b414f403a1b484b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.350705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.352757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 116/228] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:24 2022] GET /identity/v3/users/7e8efabafd7e4c729b414f403a1b484b => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:24.356236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce000f7-5e64-417f-ba5c-dd3d63b69028 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.358369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.403352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.404346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.404633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.404905 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.405594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:24.405879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.407302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.425828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 113/229] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:24.432951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.470294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:24.510464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86db5a5-2772-4157-9e4b-8e6591f9ac09 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.575671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.576313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.576590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.577953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.580364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.581827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.583762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 117/230] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:24 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:24.589766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.635138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3602-6bb7-4cab-a126-8ecc152d0efe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.668959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:24.700732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.701369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.701648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.701939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] PATH_INFO: `/v3/users/7e8efabafd7e4c729b414f403a1b484b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.703735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] RBAC: Authorizing `identity:get_user(user_id=7e8efabafd7e4c729b414f403a1b484b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.705188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.707006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 114/231] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:24 2022] GET /identity/v3/users/7e8efabafd7e4c729b414f403a1b484b => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:24.712641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.801373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944b2d5-a3a5-4de1-a310-3635e3374cc0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.816479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb0c0-dd29-4af1-aa10-04ad3d7d04ba None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:24.859671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.860306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.860583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.860854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.862921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.864366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.866075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 115/232] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:24 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:24.875148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe539532-413e-4d10-9bea-eedc7662aa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:24.875820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe539532-413e-4d10-9bea-eedc7662aa63 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:24.876096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe539532-413e-4d10-9bea-eedc7662aa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:24.876367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe539532-413e-4d10-9bea-eedc7662aa63 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/7e8efabafd7e4c729b414f403a1b484b/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:24.877863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:24.881830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe539532-413e-4d10-9bea-eedc7662aa63 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c988fc0e3664c8e8306ce7bb927936d, user_id=7e8efabafd7e4c729b414f403a1b484b, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:24.884143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe539532-413e-4d10-9bea-eedc7662aa63 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:24.907064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 118/233] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:24 2022] PUT /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/7e8efabafd7e4c729b414f403a1b484b/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:24.974296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:25.044198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cb094-4e88-452a-8023-9cd4481a765b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:25.088877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17168bd6-c7b9-4323-8947-60399ec796fa None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.089167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17168bd6-c7b9-4323-8947-60399ec796fa None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.089447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17168bd6-c7b9-4323-8947-60399ec796fa None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.091131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 119/234] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:46:25.093714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf5cf-29f1-48ce-8558-64694793df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:25.094751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf5cf-29f1-48ce-8558-64694793df5c None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.095288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf5cf-29f1-48ce-8558-64694793df5c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.095774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf5cf-29f1-48ce-8558-64694793df5c None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/7e8efabafd7e4c729b414f403a1b484b/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.096593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd9299-1660-4c9d-ae42-894b4307f412 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.096826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd9299-1660-4c9d-ae42-894b4307f412 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.097101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd9299-1660-4c9d-ae42-894b4307f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.098977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bf5cf-29f1-48ce-8558-64694793df5c None admin] RBAC: Authorizing `identity:create_grant(project_id=8c988fc0e3664c8e8306ce7bb927936d, user_id=7e8efabafd7e4c729b414f403a1b484b, role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:25.100880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bf5cf-29f1-48ce-8558-64694793df5c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:25.113443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:25.120173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 116/235] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:24 2022] PUT /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/7e8efabafd7e4c729b414f403a1b484b/roles/c374f9c914654fa8aad1636ed6f12185 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:25.142299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6bfd9299-1660-4c9d-ae42-894b4307f412 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:25.143567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6bfd9299-1660-4c9d-ae42-894b4307f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:25.231869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 120/236] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:25.487058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c6446-35b2-4492-9d72-899f8f5888c2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.487604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c6446-35b2-4492-9d72-899f8f5888c2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.487790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c6446-35b2-4492-9d72-899f8f5888c2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.489420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 117/237] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:25.495126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.495417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.495694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.517763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:25.518976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:25.528110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c6446-35b2-4492-9d72-899f8f5888c2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:25.628567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 121/238] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:25.670117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c6446-35b2-4492-9d72-899f8f5888c2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:25.683719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:25.715907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:25.716548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.716826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.717100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.729338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:25.738793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:25.739655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:25.740468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:25.744007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 118/239] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:25.749132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:25.854435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:25.897272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9183b1-a4f7-45d8-af52-cf9812bc3e56 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:25.933926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:25.933926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:25.933926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:25.934615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:25.934683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:25.934993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:25.936377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:25.949376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 122/240] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:25.955166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:25.963268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:26.139260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe32b0e-ae04-48d4-925c-b4d10921cdbb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:26.139649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175d31-14e2-4693-86e8-29c52fdff078 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:26.177156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:26.177791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.178682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.178962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] PATH_INFO: `/v3/users/83021188469f43d286d04e27ec752e01` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.187921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:26.196780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] RBAC: Authorizing `identity:get_user(user_id=83021188469f43d286d04e27ec752e01)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:26.198391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:26.200963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 119/241] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:25 2022] GET /identity/v3/users/83021188469f43d286d04e27ec752e01 => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:26.206356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:26.215571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:26.216227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.216508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.216778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.221138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:26.222687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:26.225511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 123/242] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:26.230781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:26.243285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:26.361141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581a955-7e9f-4732-a328-9819a6b82073 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:26.381206 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d1849-7e1f-4d1e-900b-af1ba3d8846c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:26.421655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:26.423174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.423837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.424914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.439866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4528f4d-da91-420d-af67-b6b96e3b7d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:26.440894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4528f4d-da91-420d-af67-b6b96e3b7d00 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.441450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4528f4d-da91-420d-af67-b6b96e3b7d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.441936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4528f4d-da91-420d-af67-b6b96e3b7d00 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.442779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4528f4d-da91-420d-af67-b6b96e3b7d00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:26.444450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4528f4d-da91-420d-af67-b6b96e3b7d00 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:26.480498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 124/243] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:26 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:46:26.481851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1094-3b36-4e1f-ba4d-53e09f2885a8 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.482187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1094-3b36-4e1f-ba4d-53e09f2885a8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.482467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1094-3b36-4e1f-ba4d-53e09f2885a8 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.483876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 125/244] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:26.490483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.490837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.491043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.533361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:26.535059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:26.538674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:26.539993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:26.570762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:26.642295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:26.667447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 120/245] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:26.673182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:26.692501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 126/246] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:26.775447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8852e-c552-47f3-8202-11d49e5f4215 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:26.805029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:26.810890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:26.811514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.811792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.812065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.813282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:26.815132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:26.817174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 121/247] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:26 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:26.823361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:26.936209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb87622-4934-4f23-8abf-450810e8b655 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:26.976367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:26.976981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:26.977259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:26.977529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:26.978288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] RBAC: Adding query filter params (role.id=2a56bd70614445349f9d810091b9f046, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=83021188469f43d286d04e27ec752e01) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:26.978597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:26.980051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:26.987975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 127/248] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:26 2022] GET /identity/v3/role_assignments?user.id=83021188469f43d286d04e27ec752e01&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=2a56bd70614445349f9d810091b9f046 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:27.006845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:27.145384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1de024-e04e-47a0-96cd-516e14698f22 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:27.187443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:27.188306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.188743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.189171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.202171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:27.207391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:27.208434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:27.209415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:27.211802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 122/249] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:27.217190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:27.284350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:27.343968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4417e2-e085-42f2-9019-298040ec1da5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:27.384474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:27.385106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.385387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.385837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.386542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:27.386848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:27.388246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:27.397402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 128/250] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:27.402939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:27.501427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:27.568705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ea4e3-2c9a-4889-b7fa-f80ae067b035 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:27.619389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:27.620030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.620310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.620586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] PATH_INFO: `/v3/users/e0f9762c2b744a9eb6c0e7b3032e6e68` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.621717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=e0f9762c2b744a9eb6c0e7b3032e6e68)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:27.623210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:27.625104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 123/251] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:27 2022] GET /identity/v3/users/e0f9762c2b744a9eb6c0e7b3032e6e68 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:27.630569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:27.745441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908892f2-fb5b-4405-b746-9ab96a0eb824 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:27.752821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:27.781506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00135781-1655-425b-9eea-5d5fd84d8818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:27.782171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135781-1655-425b-9eea-5d5fd84d8818 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.782454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135781-1655-425b-9eea-5d5fd84d8818 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.782726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135781-1655-425b-9eea-5d5fd84d8818 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.783865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00135781-1655-425b-9eea-5d5fd84d8818 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:27.785318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00135781-1655-425b-9eea-5d5fd84d8818 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:27.787528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 129/252] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:27 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:27.793000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:27.817824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af8107-3527-4517-8afc-51d236215752 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.818141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af8107-3527-4517-8afc-51d236215752 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.818420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af8107-3527-4517-8afc-51d236215752 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.820622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 130/253] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:46:27.825930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46736e-6e01-44b8-945d-db3c533cb029 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.826220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46736e-6e01-44b8-945d-db3c533cb029 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.826499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46736e-6e01-44b8-945d-db3c533cb029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.858942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ec46736e-6e01-44b8-945d-db3c533cb029 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:27.860139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec46736e-6e01-44b8-945d-db3c533cb029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:27.946192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579817ac-947e-4d9f-b6fe-a74c6fe9c6b4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:27.996394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 131/254] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:27.997071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:27.997665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823dbc2-4a0a-4d0c-b7f7-55ecacfe1e06 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:27.997971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823dbc2-4a0a-4d0c-b7f7-55ecacfe1e06 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:27.998251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823dbc2-4a0a-4d0c-b7f7-55ecacfe1e06 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:27.999665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 132/255] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:28.004987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:28.005276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:28.005552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:28.023340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:28.024020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:28.024302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:28.024574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/e0f9762c2b744a9eb6c0e7b3032e6e68/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:28.026530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] RBAC: Authorizing `identity:create_grant(project_id=8c988fc0e3664c8e8306ce7bb927936d, user_id=e0f9762c2b744a9eb6c0e7b3032e6e68, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:28.028026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:28.042769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:28.044004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:28.064652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 124/256] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:27 2022] PUT /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/e0f9762c2b744a9eb6c0e7b3032e6e68/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:28.155156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 133/257] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:28.455657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:28.520288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:28.572773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:28.594118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bd453-3e46-47e1-ae38-5368eb50721d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:28.651581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:28.652222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:28.652498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:28.652765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:28.672766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:28.678535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:28.679383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:28.680139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 125/258] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:28.686397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:28.701343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68dda-4549-42f4-858c-5579f3cd8fd2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:28.762874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:28.763573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:28.763852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:28.764167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:28.774602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:28.785489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:28.791405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:28.792294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:28.793066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 134/259] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:28.798156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:28.874709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cfc91-c660-4318-9f4d-4ac5ffc48367 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:28.959933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:28.960560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:28.960849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:28.961128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:28.961801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:28.962135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:28.963259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:28.963530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:28.979470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd58ac-c55f-4ee3-8665-a96ebedd0e6b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:28.984855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 126/260] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:28.995692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:29.056238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:29.057622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:29.062218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:29.062744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:29.063638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:29.064137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:29.065722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:29.081348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 135/261] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:29.086681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:29.113665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad2646-17b1-4745-8c0e-108cff31f0d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:29.123993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:29.151950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:29.152597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:29.152882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:29.153157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] PATH_INFO: `/v3/users/7e8efabafd7e4c729b414f403a1b484b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:29.154409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] RBAC: Authorizing `identity:get_user(user_id=7e8efabafd7e4c729b414f403a1b484b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:29.156041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:29.158115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 127/262] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:28 2022] GET /identity/v3/users/7e8efabafd7e4c729b414f403a1b484b => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:29.167053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:29.313593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3a61f-c89e-4080-9228-8a37987cf47e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:29.355192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fb055-64ae-4283-aa7f-3dafdfa10bfe None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:29.401694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:29.402706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:29.403179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:29.403643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:29.455533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:29.456544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:29.457016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:29.457492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] PATH_INFO: `/v3/users/7e8efabafd7e4c729b414f403a1b484b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:29.458878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] RBAC: Authorizing `identity:get_user(user_id=7e8efabafd7e4c729b414f403a1b484b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:29.461191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:29.463004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 136/263] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:29 2022] GET /identity/v3/users/7e8efabafd7e4c729b414f403a1b484b => generated 283 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:29.469613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:29.488502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:29.534997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:29.543017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:29.544708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:29.712960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc0421-3f77-4a9c-bab4-b3abd5cda66e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:29.783208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee4bdc-6dd7-4c36-a873-12d74816a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:29.783842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee4bdc-6dd7-4c36-a873-12d74816a99f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:29.784127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee4bdc-6dd7-4c36-a873-12d74816a99f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:29.784402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee4bdc-6dd7-4c36-a873-12d74816a99f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:29.833028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 128/264] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:29.842677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:29.935016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee4bdc-6dd7-4c36-a873-12d74816a99f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:29.936572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee4bdc-6dd7-4c36-a873-12d74816a99f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:29.988835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:30.006854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:30.055542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c76c71-c8b6-4f12-b06e-9f5fe4eba2d7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:30.109224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42690fc0-5160-4ff5-8b9c-61fc0c48ea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:30.109875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42690fc0-5160-4ff5-8b9c-61fc0c48ea1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.110179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42690fc0-5160-4ff5-8b9c-61fc0c48ea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.110454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42690fc0-5160-4ff5-8b9c-61fc0c48ea1a None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.112754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42690fc0-5160-4ff5-8b9c-61fc0c48ea1a None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:30.114221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42690fc0-5160-4ff5-8b9c-61fc0c48ea1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:30.118118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 129/265] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:29 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:30.119292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5c761-6efc-43e4-b0c3-11fb3797fe82 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.119577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5c761-6efc-43e4-b0c3-11fb3797fe82 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.119857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5c761-6efc-43e4-b0c3-11fb3797fe82 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.122371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 130/266] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:30.123095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5c761-6efc-43e4-b0c3-11fb3797fe82 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:30.139798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 137/267] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:46:30.141063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.141355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.141637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.170416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:30.171614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:30.295248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 138/268] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:30.295845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:30.319908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5c761-6efc-43e4-b0c3-11fb3797fe82 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:30.378397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:30.393214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:30.395158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.395911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.396996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.397981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=7e8efabafd7e4c729b414f403a1b484b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:30.399903 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:30.402907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a7d1b-1f5f-43a2-bb66-a6ce79023022 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:30.416731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 131/269] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:30 2022] GET /identity/v3/role_assignments?user.id=7e8efabafd7e4c729b414f403a1b484b&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:30.425673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f561833-61c2-4a45-ac43-ae90d378b568 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.426322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f561833-61c2-4a45-ac43-ae90d378b568 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.426865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f561833-61c2-4a45-ac43-ae90d378b568 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.430277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 132/270] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:30.436579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518493-0b1a-4ea9-99a2-9e3eb5474514 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.438314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518493-0b1a-4ea9-99a2-9e3eb5474514 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.438604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518493-0b1a-4ea9-99a2-9e3eb5474514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.458108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b518493-0b1a-4ea9-99a2-9e3eb5474514 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:30.459283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b518493-0b1a-4ea9-99a2-9e3eb5474514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:30.461477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:30.467892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a819af4-7fae-4e6d-b770-e0902c4f7649 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:30.519905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:30.520973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.522266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.522570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.525116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:30.526669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:30.528400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 139/271] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:30 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:30.534152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:30.544175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 133/272] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:30.597987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7283-fe4f-4cbd-86f2-24dd18c750a6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.598573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7283-fe4f-4cbd-86f2-24dd18c750a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.599048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7283-fe4f-4cbd-86f2-24dd18c750a6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.604360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 134/273] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:46:30.609558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.610115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.610656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.634037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:30.635697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:30.714470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 135/274] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:30.727728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29e782-3c12-4711-9084-6fc84f4482b9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:30.754041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:30.756713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:30.782656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:30.783348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.783643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.783921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.784664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] RBAC: Adding query filter params (role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=7e8efabafd7e4c729b414f403a1b484b) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:30.784978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:30.786661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:30.798446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 140/275] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:30 2022] GET /identity/v3/role_assignments?user.id=7e8efabafd7e4c729b414f403a1b484b&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:30.848125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:30.874321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac3a1b-7ac6-41da-9d77-167592764d53 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:30.897811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:30.933765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:30.934720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:30.935209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:30.936234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:30.950444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:30.956020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:30.956989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:30.957762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:30.959935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 136/276] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:30.965499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.111193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713316e-c522-4405-b8c4-fc289b9c27a7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.130713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1afb8-7962-462c-aaec-96601f30c4b1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.160859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.161563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.161775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.162631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.163420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:31.173957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.174573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.174858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.175136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.175809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:31.176115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.177516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.178985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:31.185960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:31.186841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:31.187719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:31.190172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 141/277] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:31.192998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 137/278] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:31.193834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.195988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.250027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:31.314967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3b857-ff80-4e37-9a97-d1a58424b35a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.316095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233ba66-ffbb-485d-be71-53e0708ee942 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.351653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.352292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.352572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.352846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.353971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.354945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.355455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.355553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.355823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.356097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.356761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:31.361710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 142/279] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:31.361829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.361931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.362007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.369818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 138/280] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:31.370630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.424396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:31.466982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163eeec-5dbe-4d09-ad37-de20c55197cd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.470647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c97d9-5bd5-480f-b28b-addabe83e6a9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.480715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:31.500823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.501447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.501727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.502030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] PATH_INFO: `/v3/users/83021188469f43d286d04e27ec752e01` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.503152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] RBAC: Authorizing `identity:get_user(user_id=83021188469f43d286d04e27ec752e01)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.503942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ba844-589a-4c23-8775-f4ec8a1edb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.504539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ba844-589a-4c23-8775-f4ec8a1edb22 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.504628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.504818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ba844-589a-4c23-8775-f4ec8a1edb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.505090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ba844-589a-4c23-8775-f4ec8a1edb22 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.505727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ba844-589a-4c23-8775-f4ec8a1edb22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.506237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 143/281] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:31 2022] GET /identity/v3/users/83021188469f43d286d04e27ec752e01 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:31.506992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.507247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ba844-589a-4c23-8775-f4ec8a1edb22 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.528514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 139/282] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:31 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:31.529741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325afd-56a4-4f7e-bcf4-594c1fedd1f8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.530066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325afd-56a4-4f7e-bcf4-594c1fedd1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.530349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3325afd-56a4-4f7e-bcf4-594c1fedd1f8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.531784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 140/283] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:31.532359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325afd-56a4-4f7e-bcf4-594c1fedd1f8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.634526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ededd12-e773-46b4-a6b1-8bffcd5bdc80 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.654114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3325afd-56a4-4f7e-bcf4-594c1fedd1f8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.668170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:31.676241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693928ec-3aac-4d1b-905b-dd3514e3f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.677118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693928ec-3aac-4d1b-905b-dd3514e3f073 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.677408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693928ec-3aac-4d1b-905b-dd3514e3f073 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.677702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693928ec-3aac-4d1b-905b-dd3514e3f073 None admin] PATH_INFO: `/v3/users/838ba28fba0943799d0c34aa7022e259` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.692878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693928ec-3aac-4d1b-905b-dd3514e3f073 None admin] RBAC: Authorizing `identity:get_user(user_id=838ba28fba0943799d0c34aa7022e259)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.694666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693928ec-3aac-4d1b-905b-dd3514e3f073 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.698255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.698899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.699182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.699460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.701098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.701592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 144/284] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:31 2022] GET /identity/v3/users/838ba28fba0943799d0c34aa7022e259 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:31.702595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.702923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.704539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.704539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.704706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 141/285] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:31 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:31.704994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.725362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:31.726614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:31.769599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:31.809530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 145/286] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:31.810386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:31.831556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e93e-9996-4f96-91c0-941958164536 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.881862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.882532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.882810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.883082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.883821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, user.id=838ba28fba0943799d0c34aa7022e259, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:31.884124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.885633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:31.898009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 142/287] 10.222.0.61 () {56 vars in 1308 bytes} [Sat Sep 24 14:46:31 2022] GET /identity/v3/role_assignments?user.id=838ba28fba0943799d0c34aa7022e259&scope.system=all&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:31.938087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d960b2-84bd-477e-9b88-a490eda47750 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:31.982068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:31.982820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:31.983107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:31.983388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/83021188469f43d286d04e27ec752e01/roles/2a56bd70614445349f9d810091b9f046` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:31.985455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c988fc0e3664c8e8306ce7bb927936d, user_id=83021188469f43d286d04e27ec752e01, role_id=2a56bd70614445349f9d810091b9f046)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:31.986962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:32.005217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 146/288] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:46:31 2022] PUT /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/users/83021188469f43d286d04e27ec752e01/roles/2a56bd70614445349f9d810091b9f046 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:32.213959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:32.265372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:32.344985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1308ee-c21d-433c-8f17-332979456024 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:32.385551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:32.386235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:32.386519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:32.386795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:32.399672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:32.405041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:32.405889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:32.406716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 143/289] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:32.412204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:32.447318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:32.542838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860f2e2-5a9e-4bba-8b5a-5ae9d7c7fb09 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:32.579014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:32.579634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:32.579913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:32.580185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:32.580854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:32.581159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:32.582640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:32.595785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 147/290] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:32.602236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:32.704615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:32.707527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346525-f8ad-4f84-a1ae-7bee4682c05c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:32.743417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:32.744213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:32.744498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:32.744825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] PATH_INFO: `/v3/users/e0f9762c2b744a9eb6c0e7b3032e6e68` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:32.746120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] RBAC: Authorizing `identity:get_user(user_id=e0f9762c2b744a9eb6c0e7b3032e6e68)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:32.747776 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:32.754059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 144/291] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:32 2022] GET /identity/v3/users/e0f9762c2b744a9eb6c0e7b3032e6e68 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:32.756409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:32.864114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:32.873494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e896a2-bff1-4a06-9cf4-0a04bdf64424 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:32.915773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:32.916415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:32.916698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:32.916974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:33.014466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:33.015947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:33.137060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:33.150657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 148/292] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:33.152803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:33.256958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1650515-5dff-4931-b768-03e9605165ef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:33.290819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b6fea-9555-4ac4-a543-aef8343ed474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:33.291452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b6fea-9555-4ac4-a543-aef8343ed474 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:33.291770 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b6fea-9555-4ac4-a543-aef8343ed474 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:33.292056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b6fea-9555-4ac4-a543-aef8343ed474 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:33.293176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b6fea-9555-4ac4-a543-aef8343ed474 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:33.294720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b6fea-9555-4ac4-a543-aef8343ed474 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:33.301220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 145/293] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:33 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:33.303238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:33.422207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3fe9f-55aa-4f3f-8bec-7071a7562ba8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:33.459553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:33.460172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:33.460456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:33.460738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:33.461461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=e0f9762c2b744a9eb6c0e7b3032e6e68) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:33.461772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:33.463247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839786a-944b-486e-9ac6-d41a7581ea01 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:33.469178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 149/294] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:33 2022] GET /identity/v3/role_assignments?user.id=e0f9762c2b744a9eb6c0e7b3032e6e68&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:34.244790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e201472-5c0e-45c2-b553-a5faab7b75cc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.245868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e201472-5c0e-45c2-b553-a5faab7b75cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.246394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e201472-5c0e-45c2-b553-a5faab7b75cc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.248235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 146/295] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:34.253747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289e4d-e31f-45e3-b469-04e722910d76 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.254295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289e4d-e31f-45e3-b469-04e722910d76 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.254788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289e4d-e31f-45e3-b469-04e722910d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.273344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e8289e4d-e31f-45e3-b469-04e722910d76 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:34.274846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e8289e4d-e31f-45e3-b469-04e722910d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:34.328552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:34.346729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 150/296] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:34.470783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea3261-43ef-4c65-bc1e-ceb6d25a0fc4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.471427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea3261-43ef-4c65-bc1e-ceb6d25a0fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.471896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea3261-43ef-4c65-bc1e-ceb6d25a0fc4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.473664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 147/297] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:34.479484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.479919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.480099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.499351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:34.500747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:34.500969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff2c2-be3b-4994-bf1d-3f007839d040 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.501501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff2c2-be3b-4994-bf1d-3f007839d040 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.502018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff2c2-be3b-4994-bf1d-3f007839d040 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.503888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 148/298] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:34.514483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.515045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.515517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.535565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:34.538714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:34.571961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 151/299] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:34.578041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:34.619340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 149/300] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:34.620228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:34.725693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d4c5c-061f-41fb-a766-668d75069b07 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:34.745451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:34.765862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333386c-7624-4a88-9369-fa099d21ee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:34.766459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333386c-7624-4a88-9369-fa099d21ee45 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.766742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333386c-7624-4a88-9369-fa099d21ee45 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.767015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333386c-7624-4a88-9369-fa099d21ee45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.777658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333386c-7624-4a88-9369-fa099d21ee45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6333386c-7624-4a88-9369-fa099d21ee45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:34.782507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:34.783437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:34.784218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 152/301] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:34.789824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:34.833151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18631-aa61-4ae6-90c8-d3e57a4d58d0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.833450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18631-aa61-4ae6-90c8-d3e57a4d58d0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.833704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18631-aa61-4ae6-90c8-d3e57a4d58d0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.836590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 153/302] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:34.840672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.840958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.841230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.860961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:34.862830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:34.896249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556665-ffbe-4548-a51c-21f5c12b56ca None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:34.930577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:34.931193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:34.931475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:34.931753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:34.932457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:34.932706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:34.934107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:34.935993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 154/303] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:34.936823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:34.945421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 150/304] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:34.946509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:34.980506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.038754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a4eae-de97-4812-9706-e7017126e9cd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.046669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.072127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a8375-5152-424b-b58b-82a1e12fe167 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.077836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.078496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.078776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.079046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.089279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:35.094127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:35.095014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:35.095796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 155/305] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:35.097143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.112556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.113188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.113460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.113730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.114842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.116904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.118712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 151/306] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:35.119229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.202844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a84ed-27e7-4366-8b7c-3e2b3f1ca698 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.222058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.228567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083292-804f-4fdf-a31d-34bf21051f48 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.236396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.237030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.237583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.237858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] PATH_INFO: `/v3/users/2fba511f9452495cafa5d3ad92fb01c0` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.250880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] RBAC: Authorizing `identity:get_user(user_id=2fba511f9452495cafa5d3ad92fb01c0)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.252639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.254289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 156/307] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/users/2fba511f9452495cafa5d3ad92fb01c0 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:35.255009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.261952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.262563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.262845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.263122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.263781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:35.264089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.265442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.274409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 152/308] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:35.275190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.282234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.356173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d1678-5701-4c48-a38f-c44a8b8b3657 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.377199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a772aa-ac2e-484d-8aa6-fb3b74705beb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.390582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.391198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.391477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.391749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] PATH_INFO: `/v3/groups` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.392379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.393820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.418339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 157/309] 10.222.0.61 () {60 vars in 1134 bytes} [Sat Sep 24 14:46:35 2022] POST /identity/v3/groups => generated 229 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:35.418339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.424072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.424752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.425056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.425300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.438853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:35.444979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:35.445996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:35.446797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 153/310] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:35.447452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.470524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.513986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.533285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6ec49-93a9-4bf4-bc5c-2146c7d2d4df None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.566109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a99cf-f58a-4b45-9e78-a8a325c4b9e7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.571897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.572720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.573143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.573574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.574569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] RBAC: Adding query filter params (role.id=2a56bd70614445349f9d810091b9f046, user.id=2fba511f9452495cafa5d3ad92fb01c0, scope.system=all) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:35.575056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.576635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.582154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 158/311] 10.222.0.61 () {56 vars in 1308 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/role_assignments?user.id=2fba511f9452495cafa5d3ad92fb01c0&scope.system=all&role.id=2a56bd70614445349f9d810091b9f046 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:35.582962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.621739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.621739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.621739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.621739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] PATH_INFO: `/v3/users/838ba28fba0943799d0c34aa7022e259` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.623506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] RBAC: Authorizing `identity:get_user(user_id=838ba28fba0943799d0c34aa7022e259)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.624992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.628234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 154/312] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/users/838ba28fba0943799d0c34aa7022e259 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:35.636411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.710562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5b1be-e4b2-415e-91cf-80cf828baf95 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.756219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.756951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.757126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.757392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.758096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:35.758399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.759793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.760985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.768870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 159/313] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:35.774413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:35.781716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cfe35-da88-4ac1-a388-bfd118b70655 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.788577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:35.825537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.826280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.826562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.826840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] PATH_INFO: `/v3/system/users/838ba28fba0943799d0c34aa7022e259/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.828251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=838ba28fba0943799d0c34aa7022e259, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.829633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.845940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 155/314] 10.222.0.61 () {58 vars in 1334 bytes} [Sat Sep 24 14:46:35 2022] PUT /identity/v3/system/users/838ba28fba0943799d0c34aa7022e259/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:35.888922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5036fe-d7bc-4486-9032-10ddb4116145 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:35.930175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:35.930950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:35.931236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:35.931511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] PATH_INFO: `/v3/users/83021188469f43d286d04e27ec752e01` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:35.932676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] RBAC: Authorizing `identity:get_user(user_id=83021188469f43d286d04e27ec752e01)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:35.934227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:35.936362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 160/315] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/users/83021188469f43d286d04e27ec752e01 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:35.943010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:36.062018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab26a2-6f55-4837-9f76-8298253f37bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:36.098099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:36.099029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:36.100092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:36.100542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:36.120961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:36.182437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:36.184181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:36.289017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 156/316] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:36.294872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:36.378142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:36.402210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6c2d1-5a6c-467c-9139-0b4e5f13910b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:36.441997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb507b56-b555-4a23-941b-8aa984ecd76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:36.442608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507b56-b555-4a23-941b-8aa984ecd76c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:36.442895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507b56-b555-4a23-941b-8aa984ecd76c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:36.443193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507b56-b555-4a23-941b-8aa984ecd76c None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:36.444314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb507b56-b555-4a23-941b-8aa984ecd76c None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:36.445833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb507b56-b555-4a23-941b-8aa984ecd76c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:36.447899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 161/317] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:36 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:36.453628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:36.509929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:36.567904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d027d-74b6-44fe-9bec-2897d07e1ba7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:36.604589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:36.605204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:36.605484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:36.605753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:36.606512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] RBAC: Adding query filter params (role.id=2a56bd70614445349f9d810091b9f046, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d, user.id=83021188469f43d286d04e27ec752e01) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:36.606820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:36.608303 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4e822-728b-4fd7-8922-fcc4a702b058 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:36.615023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 157/318] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:46:36 2022] GET /identity/v3/role_assignments?user.id=83021188469f43d286d04e27ec752e01&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=2a56bd70614445349f9d810091b9f046 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:37.670388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c12f8-8181-4eb0-b9d7-5ae05b4f9f3b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:37.670903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c12f8-8181-4eb0-b9d7-5ae05b4f9f3b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:37.670903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c12f8-8181-4eb0-b9d7-5ae05b4f9f3b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:37.672252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 162/319] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:37.678183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d803284-fd8d-49b2-891a-e2e4d453cc6d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:37.680569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d803284-fd8d-49b2-891a-e2e4d453cc6d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:37.680569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d803284-fd8d-49b2-891a-e2e4d453cc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:37.699672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7d803284-fd8d-49b2-891a-e2e4d453cc6d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:37.700865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7d803284-fd8d-49b2-891a-e2e4d453cc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:37.772452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 158/320] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:37.827968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958da29-b2f4-4bcd-972f-b660ea6f56ad None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:37.828252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958da29-b2f4-4bcd-972f-b660ea6f56ad None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:37.828696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958da29-b2f4-4bcd-972f-b660ea6f56ad None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:37.830064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 163/321] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:37.835831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1529437-47ec-45be-82aa-9aee32438989 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:37.836122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1529437-47ec-45be-82aa-9aee32438989 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:37.836398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1529437-47ec-45be-82aa-9aee32438989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:37.855187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f1529437-47ec-45be-82aa-9aee32438989 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:37.856372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f1529437-47ec-45be-82aa-9aee32438989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:37.898156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:37.930992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 159/322] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:38.036065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958da29-b2f4-4bcd-972f-b660ea6f56ad None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.046898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4d79-6486-4d27-9870-c1cd9c718d6c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.047195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4d79-6486-4d27-9870-c1cd9c718d6c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.047476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4d79-6486-4d27-9870-c1cd9c718d6c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.048851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 160/323] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:46:38.054670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.054960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.055243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.076719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:38.077935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:38.144988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958da29-b2f4-4bcd-972f-b660ea6f56ad None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.149189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 161/324] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:38.179209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.179841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.180120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.180390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.181526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.182997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.184719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 164/325] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:38.190008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.196714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.213180 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:38.298063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f8aa3-56e6-4928-9bc5-37f594738b6f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.300421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:38.304574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444646c-2bb0-4ec9-a1b8-440c15b247cc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.334923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.335529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.335811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.336081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] PATH_INFO: `/v3/groups` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.336721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.338179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.340507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.341320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.341809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.342323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.355808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 162/326] 10.222.0.61 () {60 vars in 1134 bytes} [Sat Sep 24 14:46:38 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:46:38.367756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:38.378671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:38.379645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:38.380431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:38.382257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 165/327] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:38.386492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.416342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.502922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325572-d20b-4dfe-9c68-d4b097c9d165 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.503236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:38.530373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef076976-ff7e-4165-bd3b-581d5635784a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.543456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.544078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.544366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.544641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.545312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:38.545620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.547029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.557383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 163/328] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:38.563310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.571959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.572746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.573031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.573304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.585125 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:38.590123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:38.591013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:38.591800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 166/329] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:38.594015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:38.597937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.700975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020b26-dc48-4fcd-a10e-a3104225ec4e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.725055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab9a81-4a1a-4257-b82d-617f972a0168 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.745006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.745377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.745658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.745948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] PATH_INFO: `/v3/users/2fba511f9452495cafa5d3ad92fb01c0` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.747094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] RBAC: Authorizing `identity:get_user(user_id=2fba511f9452495cafa5d3ad92fb01c0)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.748590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.751281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 164/330] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/users/2fba511f9452495cafa5d3ad92fb01c0 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:38.756966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.774513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.775136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.775422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.775990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.776730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:38.777199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.778973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.780644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:38.794128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 167/331] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:38.800130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:38.871561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:38.885338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bd7d-e80b-48b9-9ebe-5f9f5c51e646 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.908113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c6877-e364-49a1-b97e-86efed8b10bf None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:38.927845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.928514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.928868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.931095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] PATH_INFO: `/v3/system/users/2fba511f9452495cafa5d3ad92fb01c0/roles/2a56bd70614445349f9d810091b9f046` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.932676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2fba511f9452495cafa5d3ad92fb01c0, role_id=2a56bd70614445349f9d810091b9f046)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.934268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.943992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 165/332] 10.222.0.61 () {58 vars in 1334 bytes} [Sat Sep 24 14:46:38 2022] PUT /identity/v3/system/users/2fba511f9452495cafa5d3ad92fb01c0/roles/2a56bd70614445349f9d810091b9f046 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:38.955818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d008ae-b03d-42d9-b14e-fed744b0e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:38.956526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d008ae-b03d-42d9-b14e-fed744b0e30f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:38.956822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d008ae-b03d-42d9-b14e-fed744b0e30f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:38.957100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d008ae-b03d-42d9-b14e-fed744b0e30f None admin] PATH_INFO: `/v3/users/838ba28fba0943799d0c34aa7022e259` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:38.958494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d008ae-b03d-42d9-b14e-fed744b0e30f None admin] RBAC: Authorizing `identity:get_user(user_id=838ba28fba0943799d0c34aa7022e259)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:38.960880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d008ae-b03d-42d9-b14e-fed744b0e30f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:38.963101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 168/333] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/users/838ba28fba0943799d0c34aa7022e259 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:38.968793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:39.090659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e41df-6c20-48b8-82c4-bdd28a8297ea None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:39.134374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:39.135290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:39.135724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:39.136144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:39.137048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] RBAC: Adding query filter params (role.id=0f6c315ba2fd4e86ae0ef609c5a92319, user.id=838ba28fba0943799d0c34aa7022e259, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:39.137510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:39.139384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffebee7-13a6-434a-b840-4014701705bb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:39.157298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 166/334] 10.222.0.61 () {56 vars in 1308 bytes} [Sat Sep 24 14:46:38 2022] GET /identity/v3/role_assignments?user.id=838ba28fba0943799d0c34aa7022e259&scope.system=all&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:41.347002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc94d14-7e8c-40a2-98ab-75a481af5fb2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.347539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc94d14-7e8c-40a2-98ab-75a481af5fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.347539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc94d14-7e8c-40a2-98ab-75a481af5fb2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.349220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 169/335] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:41.354188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ccd3-29ec-4f2e-88a4-21e086c966a4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.355012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ccd3-29ec-4f2e-88a4-21e086c966a4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.355525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ccd3-29ec-4f2e-88a4-21e086c966a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.374291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f823-92ed-4b30-89d0-349e1809443a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.374947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f823-92ed-4b30-89d0-349e1809443a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.375447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f823-92ed-4b30-89d0-349e1809443a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.377197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 170/336] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:41.382835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3f81ccd3-29ec-4f2e-88a4-21e086c966a4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:41.383632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30a37c-bad0-4cdd-9443-b7c3634b48ec None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.384049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3f81ccd3-29ec-4f2e-88a4-21e086c966a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:41.384322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30a37c-bad0-4cdd-9443-b7c3634b48ec None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.384786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30a37c-bad0-4cdd-9443-b7c3634b48ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.402148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:41.406713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4e30a37c-bad0-4cdd-9443-b7c3634b48ec None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:41.407903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4e30a37c-bad0-4cdd-9443-b7c3634b48ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:41.476239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:41.486498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 171/337] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:41.510150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 167/338] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:41.692412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373030f5-e789-40ae-be73-cebbb1eda4d2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.693612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373030f5-e789-40ae-be73-cebbb1eda4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.694349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373030f5-e789-40ae-be73-cebbb1eda4d2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.698995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 172/339] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:41.706704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a401a-2248-4fb2-9738-b03cdd932945 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.707300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a401a-2248-4fb2-9738-b03cdd932945 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.707767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a401a-2248-4fb2-9738-b03cdd932945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.740039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1b9a401a-2248-4fb2-9738-b03cdd932945 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:41.741213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1b9a401a-2248-4fb2-9738-b03cdd932945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:41.825578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 168/340] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:41.835613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac819b5-dd19-4e23-9986-25484e238914 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.835901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac819b5-dd19-4e23-9986-25484e238914 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.836180 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac819b5-dd19-4e23-9986-25484e238914 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.841691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 173/341] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:41.846319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:41.846841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:41.847316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:41.874894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:41.876407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:41.951705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 169/342] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:41.968549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac819b5-dd19-4e23-9986-25484e238914 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.016190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.026494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.139046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eacc2-5bf4-4e03-9ca6-694de655fb3b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.164778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac819b5-dd19-4e23-9986-25484e238914 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.180759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.181655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.182136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.182604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.195051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:42.200647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:42.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:42.202744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:42.205029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 170/343] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:42.210421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.216352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.216994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.217261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.217525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.247251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:42.252108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:42.253168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:42.254141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 174/344] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:42.260224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.290308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.329675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b361b-dc96-4155-9ea7-9e2bd99a0db1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.364831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.370994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.372901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.373504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.374047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.374865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:42.375330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:42.377009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:42.388221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 171/345] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:42.389804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.418411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642adf26-2043-4f1e-bca5-1401f2420209 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.455087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.456534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.457034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.457499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.458412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:42.458937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:42.460535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:42.470392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 175/346] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:42.471552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.515255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30635b-b1a9-4a2c-a8d0-159e1e67b2fd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.546882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.558082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.559116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.559595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.560974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.575427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:42.582006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:42.582891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:42.583750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:42.587964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 172/347] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:42.593945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.605472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f0900-72a7-41f9-8049-616ab81ae9c7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.625694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.650077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.650719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.651054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.651331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.666657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:42.673238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:42.674314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:42.675110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 176/348] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:42.675765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:42.730211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026892f8-2704-41c1-a883-5b4eded3b6b9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.773391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.774349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.774839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.775296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.810550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6785b4f-667c-4f88-9f2b-5814494965fb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:42.850619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.862135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:42.863033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:42.863315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:42.863588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:42.869440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:42.871273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:42.923214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:42.953447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:42.955385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.013599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 173/349] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:43.014823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.091304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 177/350] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:43.092500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.121052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.135608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f399a38-5410-4b62-8009-ef5169c9d7fb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.178675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.184025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.184959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.185443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.186181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.187374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:43.187884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.189465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589f2f8-3a51-414a-9c75-f88ccafdd4f7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.202661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 174/351] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.202661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557cec3-fae2-4217-8e29-83784cc24976 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.202883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557cec3-fae2-4217-8e29-83784cc24976 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.203154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557cec3-fae2-4217-8e29-83784cc24976 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.204548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 175/352] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:43.206514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557cec3-fae2-4217-8e29-83784cc24976 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.211407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fd40f-60f7-422d-8c4a-ef0e0f044a4a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.258471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.259358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.259785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.260225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.260900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:43.261242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.262633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.275381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 178/353] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.275991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.341275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557cec3-fae2-4217-8e29-83784cc24976 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.378558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.379174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.379473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.379749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.382344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.383799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.386003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 176/354] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.387592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.398924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80587dee-91a9-4e9c-975d-ff6a0bfe9443 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.410077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.438179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0a538-b06d-49ce-818d-914f957cbc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.438820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a538-b06d-49ce-818d-914f957cbc78 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.439095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a538-b06d-49ce-818d-914f957cbc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.439364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a538-b06d-49ce-818d-914f957cbc78 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.440499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0a538-b06d-49ce-818d-914f957cbc78 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.441973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0a538-b06d-49ce-818d-914f957cbc78 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.443735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 179/355] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.444864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.445132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.445426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.463123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.469554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:43.471013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:43.529885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52ab39-a2e0-44f3-b0d8-25010901867b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.548502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 180/356] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:43.549471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.575603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.576544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.577048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.577519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.660910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.662783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.684415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.684592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09753a26-f8d5-4df7-905c-b5858eeb876f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.732812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.733492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.733776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.734099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] PATH_INFO: `/v3/users/2fba511f9452495cafa5d3ad92fb01c0` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.735952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] RBAC: Authorizing `identity:get_user(user_id=2fba511f9452495cafa5d3ad92fb01c0)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.737992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.745705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 181/357] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/users/2fba511f9452495cafa5d3ad92fb01c0 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.746636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.750388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.793867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 177/358] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:43.795124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.875553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8acb59-0235-4e99-ab51-093f72debf0a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.912287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72fb9e-e91f-4177-945f-51c3bdf9c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.912916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fb9e-e91f-4177-945f-51c3bdf9c167 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.913190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fb9e-e91f-4177-945f-51c3bdf9c167 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.913458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72fb9e-e91f-4177-945f-51c3bdf9c167 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.922414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8af54f-df07-4fa4-9567-217e1271c554 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:43.924146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72fb9e-e91f-4177-945f-51c3bdf9c167 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.925845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72fb9e-e91f-4177-945f-51c3bdf9c167 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.929761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 182/359] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.929761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af21800-3618-489e-afac-3cdffc8047e4 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.929761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af21800-3618-489e-afac-3cdffc8047e4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.929761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af21800-3618-489e-afac-3cdffc8047e4 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.930903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 183/360] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:43.931614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af21800-3618-489e-afac-3cdffc8047e4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:43.949195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:43.970845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:43.971487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:43.971767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:43.972038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:43.973129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:43.975135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:43.976748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 178/361] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:43.977511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.010416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:44.058566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af21800-3618-489e-afac-3cdffc8047e4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.096687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d5458-3362-4f52-b103-f6f1596b4e4a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.097752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.098617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.098898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.099694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.100429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] RBAC: Adding query filter params (role.id=2a56bd70614445349f9d810091b9f046, user.id=2fba511f9452495cafa5d3ad92fb01c0, scope.system=all) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:44.100742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.102209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.117525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 184/362] 10.222.0.61 () {56 vars in 1308 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/role_assignments?user.id=2fba511f9452495cafa5d3ad92fb01c0&scope.system=all&role.id=2a56bd70614445349f9d810091b9f046 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:44.118201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.135649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.135649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.135649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.135649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.136709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.138266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.140328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 179/363] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:43 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:44.142143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.235067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:44.256294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ef548-b45a-4b7c-9292-eeaeb5b8fc97 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.303801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:44.313477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35beec7-9d4c-4a5a-897d-53a1c9a0fe1a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.316331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81afb-8ad4-407e-9acb-9879a36c304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.316980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81afb-8ad4-407e-9acb-9879a36c304a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.317262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81afb-8ad4-407e-9acb-9879a36c304a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.317535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81afb-8ad4-407e-9acb-9879a36c304a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.329689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d81afb-8ad4-407e-9acb-9879a36c304a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-21d81afb-8ad4-407e-9acb-9879a36c304a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:44.335372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:44.336560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 185/364] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:44.337620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.338374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.338374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.357004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:44.359848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:44.361152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.361997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.362680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.363080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.363967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:44.364414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.366672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.373000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 180/365] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:44.374132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.451160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 186/366] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:44.452745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.509236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc51d59-dccc-4a90-9c8e-cb4ba8380f57 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.530964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:44.559904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.560955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.561444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.561979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.562943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:44.563460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.565123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.573690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 181/367] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:44.574583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.593121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59237c-5402-4fdb-b6dd-86759a9f4089 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.613885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:44.638903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.639713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.640087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.640437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.641744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.643976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.648769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 187/368] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:44.653238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.777510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a21437-665c-4d63-89ed-92fe61c380fa None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.796995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7af87-6300-440f-a25b-b69ba71cb363 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:44.819555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.820218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.820570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.820919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.821747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:44.822156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.823695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.834057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 188/369] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:44.835176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.837124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:44.838014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:44.838464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:44.838944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:44.839823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:44.840301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:44.841944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:44.852800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 182/370] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:44.859201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:44.931909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:44.954283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:45.036952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffc82-bb57-4807-bd88-6ad327061213 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.066584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015ab38-089e-4367-a4b3-c6ea3b5c0b2b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.094471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.095023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.095311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.095597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.155199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.155855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.156143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.156414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.175705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:45.177239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:45.186434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:45.191823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:45.192772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:45.193694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 189/371] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:45.202525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:45.227520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:45.294025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 183/372] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:46:45.300729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:45.398258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb214e-f6d5-4509-9f02-d4cf680bd4f7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.408122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:45.434402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730615a3-b745-4b04-a391-24ec538bb850 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.446543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.447179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.447467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.447742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.449048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:45.449364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:45.451651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:45.464480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 190/373] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:45.470777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:45.486801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.488013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.488477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.488940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.490305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:45.491937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:45.494130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 184/374] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:45 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:45.501651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:45.529077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:45.600084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ce593-d0b6-4c97-ac95-6ee298af8dbc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.642389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.643900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.644185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.644461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.722287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67502f8-6db4-4251-9f7e-dba102cac221 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.734181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:45.767139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.767805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.768089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.768488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.769647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:45.771758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:45.773647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 185/375] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:45 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:45.779466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:45.826973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:45.828439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:45.871004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:45.893700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10af98-e63e-44d0-a6ef-4a83bc28d289 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:45.946474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:45.947210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:45.947489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:45.947867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:45.948614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:45.948929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:45.950422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:45.957308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 186/376] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:45 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:46.000866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 191/377] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:46.007123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:46.132982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93b3ea-62ca-4f52-ae6c-35ab203bd9eb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:46.175423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:46.176082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:46.176366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:46.176640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] PATH_INFO: `/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:46.179602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] RBAC: Authorizing `identity:get_project(project_id=54d7e41cbbc346c5a0a397fdcad9ecdd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:46.181077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:46.182855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 187/378] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:46 2022] GET /identity/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:46.188488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:46.271442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:46.312081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e343c-7db1-4ec4-8959-24e361c46d6f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:46.357634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d59d5b-f64f-46ba-8cc7-68fad2133b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:46.358328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59d5b-f64f-46ba-8cc7-68fad2133b13 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:46.358614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59d5b-f64f-46ba-8cc7-68fad2133b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:46.358889 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59d5b-f64f-46ba-8cc7-68fad2133b13 None admin] PATH_INFO: `/v3/groups/8ed0aa8355fe4daaaca1f7ac96d31367` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:46.368248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d59d5b-f64f-46ba-8cc7-68fad2133b13 None admin] RBAC: Authorizing `identity:get_group(group_id=8ed0aa8355fe4daaaca1f7ac96d31367)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:46.369712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d59d5b-f64f-46ba-8cc7-68fad2133b13 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:46.371381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 192/379] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:46 2022] GET /identity/v3/groups/8ed0aa8355fe4daaaca1f7ac96d31367 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:46.377414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:46.408029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:46.529015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76971d-a43c-4ff4-8085-cd8bc6422a90 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:46.569528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:46.570326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:46.570489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:46.570728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:46.571457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] RBAC: Adding query filter params (group.id=8ed0aa8355fe4daaaca1f7ac96d31367, role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=54d7e41cbbc346c5a0a397fdcad9ecdd) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:46.571768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:46.573194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fed3e-f756-49e4-87f4-e2863dd352b4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:46.579182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 188/380] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:46 2022] GET /identity/v3/role_assignments?group.id=8ed0aa8355fe4daaaca1f7ac96d31367&scope.project.id=54d7e41cbbc346c5a0a397fdcad9ecdd&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:47.245799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9d255-a482-4465-baed-c81d7b570845 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.246353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9d255-a482-4465-baed-c81d7b570845 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.246443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9d255-a482-4465-baed-c81d7b570845 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.247866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 193/381] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:47.253119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b2104-2a92-4075-974d-258f2bb36e89 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.253409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b2104-2a92-4075-974d-258f2bb36e89 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.253687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b2104-2a92-4075-974d-258f2bb36e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.273945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-420b2104-2a92-4075-974d-258f2bb36e89 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:47.275166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-420b2104-2a92-4075-974d-258f2bb36e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:47.364085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 189/382] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:47.576064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896abd8f-38bc-453b-bf85-a696b2aa736b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.576411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896abd8f-38bc-453b-bf85-a696b2aa736b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.576585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896abd8f-38bc-453b-bf85-a696b2aa736b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.578036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 194/383] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:47.583459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.583904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.583982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.598688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e6f56-fdb5-447e-b3cb-93fbfaa42b22 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.599306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e6f56-fdb5-447e-b3cb-93fbfaa42b22 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.599765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e6f56-fdb5-447e-b3cb-93fbfaa42b22 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.600665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:47.601752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 195/384] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:47.606949 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.607555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.607929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:47.608264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.609187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:47.640018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:47.642361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:47.716954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 190/385] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:47.717940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:47.734825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 196/386] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:47.828085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283dbdf-1602-4207-a846-f70acd66d6dd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:47.864095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:47.864790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:47.865079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:47.865359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:47.868014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:47.876838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:47.882446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:47.883297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:47.884079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:47.888475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 191/387] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:47.895011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.002576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.022252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.102630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ccf07-cb41-404b-8889-29baec4fe2fd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.120972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf7d74-da33-46a9-837f-c4bd1e6a63c4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.135436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.136048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.136327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.136599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.137256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:48.137557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.139030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.147805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 197/388] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.148608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.169024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.170172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.170598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.170994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.185986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:48.196315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:48.197222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:48.198010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 192/389] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:48.199691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.261618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f5a72-5308-4469-81cb-feb715e854c8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.274380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.299200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.311910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.313037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.313410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.313776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.328103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37b1cc-48fe-415f-ae27-0b0ba0ee642a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.328518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:48.335727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:48.336695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:48.337473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:48.342071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 198/390] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:48.344340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.372885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.373491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.373778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.374100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.375360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.376913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.378664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 193/391] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.381188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.464798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e4032-2158-4cf4-b502-8ead93967ad7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.501845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.502510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.502792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.503067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.503738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:48.504043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.505481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.514401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 199/392] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.514707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea641-964b-42bb-afd3-e15d5d0288b2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.515340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.546096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.560980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.561638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.561968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.562653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.563323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:48.563628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.565242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.569671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.581982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 194/393] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.582989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.632200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c791b51-ff76-46fd-9c46-4d5208b36763 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.670132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.671137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.671619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.672121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.673607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.675331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.682667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 200/394] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.684008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.715420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36c833-3868-44ce-ae2c-d28d74ae8f7a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.767979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.768613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.768896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.769172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.771778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.773201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.775161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 195/395] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.775980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.805536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc57ae-7608-45ff-9a9a-74f881f540c6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.824344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.845038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.845855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.846172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.846453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.847219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:48.847580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.849198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.851080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 201/396] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:48.852159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.900758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5259bf-0c4c-48f8-a640-80a926548796 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:48.940656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4f378-0c2c-46d1-867b-5f962d699ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:48.941328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4f378-0c2c-46d1-867b-5f962d699ce4 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.941614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4f378-0c2c-46d1-867b-5f962d699ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.941920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4f378-0c2c-46d1-867b-5f962d699ce4 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.943998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4f378-0c2c-46d1-867b-5f962d699ce4 None admin] RBAC: Authorizing `identity:create_grant(project_id=16631e93412a46bf9ab3e96da45f3ed6, group_id=4d3f6e873e5f4b6fa049c1215b9d15c4, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:48.945404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4f378-0c2c-46d1-867b-5f962d699ce4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:48.959336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 196/397] 10.222.0.61 () {58 vars in 1442 bytes} [Sat Sep 24 14:46:48 2022] PUT /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:48.960846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376ec2d-abf3-4606-a9c6-ab1b41271fb0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:48.961210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376ec2d-abf3-4606-a9c6-ab1b41271fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:48.961581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376ec2d-abf3-4606-a9c6-ab1b41271fb0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:48.963215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 197/398] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:48.968396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376ec2d-abf3-4606-a9c6-ab1b41271fb0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:48.975428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37980ee-986c-4c45-a50b-a2e13816239e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:49.035448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd6df-7fba-416a-97c5-f043da548bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:49.037464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd6df-7fba-416a-97c5-f043da548bac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.038006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd6df-7fba-416a-97c5-f043da548bac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.038779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd6df-7fba-416a-97c5-f043da548bac None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.041151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bd6df-7fba-416a-97c5-f043da548bac None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:49.043666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bd6df-7fba-416a-97c5-f043da548bac None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:49.046567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 202/399] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:49.048073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.048531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.048998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.073536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:49.075403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:49.101110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376ec2d-abf3-4606-a9c6-ab1b41271fb0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:49.118345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:49.143208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:49.144009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.144300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.144576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.145307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:49.148135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:49.149327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:49.152414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 198/400] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:48 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:49.153613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:49.170293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 203/401] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:49.171308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:49.284860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b256a-728e-45c1-8660-162dc3196b50 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:49.299418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130fcd3-561b-4b9d-b4f7-cd21a91876f0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:49.321712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17be6-5994-43b0-9e8d-d52407de2ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:49.322477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17be6-5994-43b0-9e8d-d52407de2ed6 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.322864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17be6-5994-43b0-9e8d-d52407de2ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.323153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17be6-5994-43b0-9e8d-d52407de2ed6 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.325419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f17be6-5994-43b0-9e8d-d52407de2ed6 None admin] RBAC: Authorizing `identity:create_grant(project_id=16631e93412a46bf9ab3e96da45f3ed6, group_id=4d3f6e873e5f4b6fa049c1215b9d15c4, role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:49.326990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f17be6-5994-43b0-9e8d-d52407de2ed6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:49.341834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 199/402] 10.222.0.61 () {58 vars in 1442 bytes} [Sat Sep 24 14:46:49 2022] PUT /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/c374f9c914654fa8aad1636ed6f12185 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:49.343289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916c027-93f9-4453-924e-11314cf2c86a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.343483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:49.343647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916c027-93f9-4453-924e-11314cf2c86a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.344026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916c027-93f9-4453-924e-11314cf2c86a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.344655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.345124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.350139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 200/403] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:49.350354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.352039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.352518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.352885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.353472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c988fc0e3664c8e8306ce7bb927936d, group_id=4d3f6e873e5f4b6fa049c1215b9d15c4, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:49.355115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:49.376372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 204/404] 10.222.0.61 () {58 vars in 1442 bytes} [Sat Sep 24 14:46:49 2022] PUT /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:49.386533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:49.388329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:49.475712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:49.528829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 201/405] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:49.611222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:49.657769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:49.719944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13224e62-5d95-4313-93fd-6d67735c96a3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:49.776261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:49.777228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:49.777697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:49.778298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:49.795161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:49.805540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:49.806428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:49.807218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:49.822369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:49.829659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 205/406] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:49.842820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.001050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f12fb-fbe6-4d93-9b46-577038d357c6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.035619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9d645-4c95-45f3-8c53-3eae38b3d990 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.037855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:50.074881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.075913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.076410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.076981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.090188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.090807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.091087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.091353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.092018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:50.092316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.092417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:50.093681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.102541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 206/407] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:50.102696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:50.103974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:50.104788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 202/408] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:50.109178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.112022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.178235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:50.297485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b521d9-f393-4712-bcdc-8dae8aa43bc0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.312926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ff5c1-b57f-4765-8ecf-77a60ef012da None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.364495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.365706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.365944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.366219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.367900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.370721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.372523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 207/409] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:50 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:50.380003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.386063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.386680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.386958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.387230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.387899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:50.388202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.389634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.409351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 203/410] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:50.415158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.446269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:50.586569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced207c6-f719-45cc-a581-c05f8490b970 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.614919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:50.625187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b780c-b39b-40f9-8e53-2f5a1b2ba72d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.658640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.659298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.659586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.659865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.661039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.662562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.664516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 208/411] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:50 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:50.669706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.672178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.673091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.673364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] PATH_INFO: `/v3/groups/8ed0aa8355fe4daaaca1f7ac96d31367` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.674502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] RBAC: Authorizing `identity:get_group(group_id=8ed0aa8355fe4daaaca1f7ac96d31367)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.675941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.677944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 204/412] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:50 2022] GET /identity/v3/groups/8ed0aa8355fe4daaaca1f7ac96d31367 => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:50.687723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.851103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe158-89cb-42d9-add4-42358f074bbc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.882935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:50.886364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12037237-6bb4-460a-b871-f0bcb619d588 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:50.889528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.890228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.890514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.890789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] PATH_INFO: `/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.892071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] RBAC: Authorizing `identity:get_project(project_id=54d7e41cbbc346c5a0a397fdcad9ecdd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.894382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.896626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 205/413] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:50 2022] GET /identity/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:50.902521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:50.954089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f66009-2e93-4883-9782-81c36bbe7613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:50.954755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66009-2e93-4883-9782-81c36bbe7613 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:50.955028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66009-2e93-4883-9782-81c36bbe7613 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:50.955296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f66009-2e93-4883-9782-81c36bbe7613 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:50.961099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f66009-2e93-4883-9782-81c36bbe7613 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c988fc0e3664c8e8306ce7bb927936d, group_id=4d3f6e873e5f4b6fa049c1215b9d15c4, role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:50.962586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f66009-2e93-4883-9782-81c36bbe7613 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:50.968164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:50.993698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 209/414] 10.222.0.61 () {58 vars in 1442 bytes} [Sat Sep 24 14:46:50 2022] PUT /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d/groups/4d3f6e873e5f4b6fa049c1215b9d15c4/roles/c374f9c914654fa8aad1636ed6f12185 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:51.047018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b483314-8d36-4e1a-b363-993bede10c87 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:51.083300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52649ca-32b5-4adc-bec7-cdc059789be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:51.084552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52649ca-32b5-4adc-bec7-cdc059789be9 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:51.084831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52649ca-32b5-4adc-bec7-cdc059789be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:51.085100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52649ca-32b5-4adc-bec7-cdc059789be9 None admin] PATH_INFO: `/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd/groups/8ed0aa8355fe4daaaca1f7ac96d31367/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:51.087092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52649ca-32b5-4adc-bec7-cdc059789be9 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d7e41cbbc346c5a0a397fdcad9ecdd, group_id=8ed0aa8355fe4daaaca1f7ac96d31367, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:51.088607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52649ca-32b5-4adc-bec7-cdc059789be9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:51.112063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 206/415] 10.222.0.61 () {58 vars in 1442 bytes} [Sat Sep 24 14:46:50 2022] PUT /identity/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd/groups/8ed0aa8355fe4daaaca1f7ac96d31367/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:51.376035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726105f-ad5b-4218-861b-a27beb5ba87d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:51.376693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726105f-ad5b-4218-861b-a27beb5ba87d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:51.376693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726105f-ad5b-4218-861b-a27beb5ba87d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:51.378024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 210/416] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:51.384158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b27af-3240-47b1-9219-d37535be8d0c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:51.384727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b27af-3240-47b1-9219-d37535be8d0c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:51.385200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b27af-3240-47b1-9219-d37535be8d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:51.404365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e6b27af-3240-47b1-9219-d37535be8d0c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:51.405885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e6b27af-3240-47b1-9219-d37535be8d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:51.481403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 207/417] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:51.819849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726105f-ad5b-4218-861b-a27beb5ba87d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:51.962720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726105f-ad5b-4218-861b-a27beb5ba87d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:52.011490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:52.012142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.012428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.012705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.028388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:52.033743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:52.034651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:52.035502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:52.037639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 211/418] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:52.042647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420402-41e8-4d75-8a68-5f4b8d735f06 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.043089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420402-41e8-4d75-8a68-5f4b8d735f06 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.043258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420402-41e8-4d75-8a68-5f4b8d735f06 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.044546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 208/419] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:52.045580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:52.049393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c05ef-966d-46a3-b8c0-32c82f15ae0f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.049678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c05ef-966d-46a3-b8c0-32c82f15ae0f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.049985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c05ef-966d-46a3-b8c0-32c82f15ae0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.077622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-284c05ef-966d-46a3-b8c0-32c82f15ae0f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:52.078910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-284c05ef-966d-46a3-b8c0-32c82f15ae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:52.086260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.106590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.166329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 209/420] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:52.167675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846bb8f-69e0-4df9-bd82-2eaf93ad548d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.167972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846bb8f-69e0-4df9-bd82-2eaf93ad548d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.168251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846bb8f-69e0-4df9-bd82-2eaf93ad548d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.169501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 210/421] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:52.174816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.175099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.175374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.212472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:52.214012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:52.225759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7eabf-af44-4b92-b73a-7a4d0abb1f22 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:52.285337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:52.286065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.286348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.286687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.287362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:52.287668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:52.289053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:52.305016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 212/422] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:52.312044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:52.345611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 211/423] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:46:52.421916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.447825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b8c9f-94df-4a0e-a53a-0a3033674a00 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:52.497853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:52.498773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.499199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.499638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.576582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:52.578303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:52.626698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:52.697806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.726826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 213/424] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:52.732155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.733656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:52.757434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268d91e-9ac8-433a-ba4d-65605474c8c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:52.796221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:52.797091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.797544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.798009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.809476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:52.814930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:52.815864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:52.816657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:52.824470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 212/425] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:46:52.825654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:52.872561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1f4ae-bfc4-4acf-a122-10e746ef3e5b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:52.909457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:52.910014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.910292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.910563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.912281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:52.913825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:52.915750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 214/426] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:52 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:52.916627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:52.938080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e468937-ea86-4a86-9ffa-b87f4870642f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:52.973793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.976251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:52.977820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:52.978444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:52.979099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:52.989117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:52.994201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:52.999544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:53.000626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:46:53.001410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:53.002022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 213/427] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:53.002994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.031866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2301fa-863b-469e-8250-cfadd02bed45 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.069102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.070064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.070562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.071057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.072010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:53.072609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.074200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.083428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 215/428] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:53.085029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.121646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240828e3-4745-44b7-93d5-454789764b4c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.160068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.160680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.160966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.161247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.163076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.164588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.166749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 214/429] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:53.167607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.193095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b2144-08fc-4f40-8642-87710891dc90 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.207259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:53.230916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.232013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.232489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.233087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.233964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:53.234496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.236281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.246476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 216/430] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:53.247617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.248435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:53.291685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf464d-cbc7-46d7-b5fd-786eaf6bf378 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.333821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.334472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.334757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.335038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.358602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cdad7-80dc-409d-892e-5faa6d79b58f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.395750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.396803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.397317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.397779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.398718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:53.399324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.401050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.407818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 217/431] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:53.408934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.413991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.415521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.458018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:53.502068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:53.532151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284cc18-879a-4954-b59d-88d0891c15d4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.543259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 215/432] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:53.550017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.582143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f915d-0acb-4ac6-a242-ad5cf68134fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.582782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f915d-0acb-4ac6-a242-ad5cf68134fd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.583064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f915d-0acb-4ac6-a242-ad5cf68134fd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.583906 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f915d-0acb-4ac6-a242-ad5cf68134fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.666552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f915d-0acb-4ac6-a242-ad5cf68134fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.667950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f915d-0acb-4ac6-a242-ad5cf68134fd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.677843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2b8e1-12c8-423d-b536-6bfc9e376c00 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.713176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:53.728495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb7ae3-14d2-4188-9860-bde6acf537a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.729563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb7ae3-14d2-4188-9860-bde6acf537a6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.729871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb7ae3-14d2-4188-9860-bde6acf537a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.730178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb7ae3-14d2-4188-9860-bde6acf537a6 None admin] PATH_INFO: `/v3/projects/16631e93412a46bf9ab3e96da45f3ed6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.731333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb7ae3-14d2-4188-9860-bde6acf537a6 None admin] RBAC: Authorizing `identity:get_project(project_id=16631e93412a46bf9ab3e96da45f3ed6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.732789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb7ae3-14d2-4188-9860-bde6acf537a6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.739242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 216/433] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/projects/16631e93412a46bf9ab3e96da45f3ed6 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:53.741108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b06f1e-15c8-4c5c-9ec3-5680d2aa1bd7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.741396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b06f1e-15c8-4c5c-9ec3-5680d2aa1bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.741675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b06f1e-15c8-4c5c-9ec3-5680d2aa1bd7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.743210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 217/434] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:53.744340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b06f1e-15c8-4c5c-9ec3-5680d2aa1bd7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.794305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 218/435] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:46:53.795579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.795864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.796193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.809292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:53.818960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:53.821172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:53.889623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b06f1e-15c8-4c5c-9ec3-5680d2aa1bd7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:53.894739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 219/436] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:53.895589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.931842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99930a29-97e3-4011-8dcc-b9034a6122eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:53.932516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99930a29-97e3-4011-8dcc-b9034a6122eb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.932764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99930a29-97e3-4011-8dcc-b9034a6122eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.932929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99930a29-97e3-4011-8dcc-b9034a6122eb None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.934214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99930a29-97e3-4011-8dcc-b9034a6122eb None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:53.935643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99930a29-97e3-4011-8dcc-b9034a6122eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:53.937981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 218/437] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:53.940242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc0ac3-4056-4334-b5a4-247320ef2d52 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:53.940674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc0ac3-4056-4334-b5a4-247320ef2d52 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:53.941085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc0ac3-4056-4334-b5a4-247320ef2d52 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:53.942480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 219/438] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:53.943252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc0ac3-4056-4334-b5a4-247320ef2d52 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:53.979830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.019708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2777ff1-ebbf-4614-87e9-687b2cbfce1c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.055773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4ec40-b608-463c-83cf-63575f2ce6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.056404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4ec40-b608-463c-83cf-63575f2ce6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.056688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4ec40-b608-463c-83cf-63575f2ce6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.057011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4ec40-b608-463c-83cf-63575f2ce6d8 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.058083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4ec40-b608-463c-83cf-63575f2ce6d8 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.059487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4ec40-b608-463c-83cf-63575f2ce6d8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.061379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 220/439] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:54.062499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.062777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.063047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.071446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc0ac3-4056-4334-b5a4-247320ef2d52 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.074262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.080043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:54.081206 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:54.111411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.112580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.113054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.113464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.114363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=16631e93412a46bf9ab3e96da45f3ed6) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:54.115358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.116907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.123193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 220/440] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:53 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=16631e93412a46bf9ab3e96da45f3ed6&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:54.124539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.172061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 221/441] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:54.172997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.246547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36fa7d-be2c-4669-94cd-333b506c929f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.264712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.283826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.284471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.284756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.285283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.286437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.287939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.294634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 221/442] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:54.295945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.317261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf34e77-9685-4dbf-8107-00322f005674 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.339258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.359964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.360606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.360887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.361165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.375768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:54.381694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:54.382645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:54.383454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 222/443] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:54.390888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.419275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4cb1a-44a8-4c2a-b7a9-975e93444725 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.459550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.460161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.460441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.460715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.461454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=0f6c315ba2fd4e86ae0ef609c5a92319, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:54.461766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.463658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.470761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 222/444] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=0f6c315ba2fd4e86ae0ef609c5a92319 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:54.471475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.528078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5890f1e-bae5-4049-a22a-2a045c710ce4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.562723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.573834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.574494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.574766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.575152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.575975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:54.576291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.577719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.590442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 223/445] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:54.596259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.617222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.624857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8d36-b5a3-4eaf-9b21-111215a6548e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.672263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.673517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.674020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.674458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.687359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:46:54.693781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:54.694893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:54.695870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 223/446] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:54.700848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.720661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeac38-4423-4de4-8bd4-a536af2cb38c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.765414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.768764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.768764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.768764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.825478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb9bc7-e3ee-4086-a054-332370f637ad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.834153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.849539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.851151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.866169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.866775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.867052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.867319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.867983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:54.868286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.869660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.878728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 224/447] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:54.884449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.910409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.960749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 224/448] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:46:54.966332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.989697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bb35c-8bc5-4cdf-8848-65fdd631132f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.022021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.022626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.022908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.023187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.049662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:55.065773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9f3d6-0b55-4fd0-ac89-d2071d8952dc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.091883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.093269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.099592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.100218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.100499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.100771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] PATH_INFO: `/v3/projects/8c988fc0e3664c8e8306ce7bb927936d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.101876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] RBAC: Authorizing `identity:get_project(project_id=8c988fc0e3664c8e8306ce7bb927936d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.103317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.104919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 225/449] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/projects/8c988fc0e3664c8e8306ce7bb927936d => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:55.110386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:55.120595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:55.195503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 225/450] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:46:55.200976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:55.207614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b789793-57e6-4ad4-b5de-24931b0272f3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.240865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.241482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.241766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.242062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] PATH_INFO: `/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.243169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] RBAC: Authorizing `identity:get_group(group_id=4d3f6e873e5f4b6fa049c1215b9d15c4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.244635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.246503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 226/451] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:55 2022] GET /identity/v3/groups/4d3f6e873e5f4b6fa049c1215b9d15c4 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:55.251856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:55.275100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:55.300861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a6757-4e07-469e-9bfe-aeed948c252f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.323465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:55.334382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.335000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.335276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.335547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] PATH_INFO: `/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.336648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] RBAC: Authorizing `identity:get_project(project_id=54d7e41cbbc346c5a0a397fdcad9ecdd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.338106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.339706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 226/452] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:46:55 2022] GET /identity/v3/projects/54d7e41cbbc346c5a0a397fdcad9ecdd => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:55.345214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:55.347862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e28aa3-5a25-41c6-a85a-8a981e3e6330 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.383980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.384581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.384868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.385145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.385859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] RBAC: Adding query filter params (group.id=4d3f6e873e5f4b6fa049c1215b9d15c4, role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=8c988fc0e3664c8e8306ce7bb927936d) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:55.386205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.387634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.393173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 227/453] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:55 2022] GET /identity/v3/role_assignments?group.id=4d3f6e873e5f4b6fa049c1215b9d15c4&scope.project.id=8c988fc0e3664c8e8306ce7bb927936d&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:55.446841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5df2b-0fa4-4042-b1fc-062357912efb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.482884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad5884-4cc4-4932-bd6d-645499274f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.483522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad5884-4cc4-4932-bd6d-645499274f4a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.483800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad5884-4cc4-4932-bd6d-645499274f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.484070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad5884-4cc4-4932-bd6d-645499274f4a None admin] PATH_INFO: `/v3/groups/8ed0aa8355fe4daaaca1f7ac96d31367` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.485207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad5884-4cc4-4932-bd6d-645499274f4a None admin] RBAC: Authorizing `identity:get_group(group_id=8ed0aa8355fe4daaaca1f7ac96d31367)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.486736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad5884-4cc4-4932-bd6d-645499274f4a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.488591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 227/454] 10.222.0.61 () {56 vars in 1180 bytes} [Sat Sep 24 14:46:55 2022] GET /identity/v3/groups/8ed0aa8355fe4daaaca1f7ac96d31367 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:55.495210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:55.586986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:55.601164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b169-4a74-4c8b-b97e-15f88d9b022a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:55.636224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:55.636839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:55.637120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:55.637395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:55.638173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] RBAC: Adding query filter params (group.id=8ed0aa8355fe4daaaca1f7ac96d31367, role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=54d7e41cbbc346c5a0a397fdcad9ecdd) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:55.638479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:55.639919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c692f-ba60-4dd8-bbc1-d7ce6e708f5e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:55.645624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 228/455] 10.222.0.61 () {56 vars in 1376 bytes} [Sat Sep 24 14:46:55 2022] GET /identity/v3/role_assignments?group.id=8ed0aa8355fe4daaaca1f7ac96d31367&scope.project.id=54d7e41cbbc346c5a0a397fdcad9ecdd&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:58.285180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3a883-cb3a-498e-9746-fdc67ec45acf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.285717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3a883-cb3a-498e-9746-fdc67ec45acf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.285717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3a883-cb3a-498e-9746-fdc67ec45acf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.287991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 228/456] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:58.292887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4b3b-0615-4c56-8f25-70ac8500a3cd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.293288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4b3b-0615-4c56-8f25-70ac8500a3cd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.293451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4b3b-0615-4c56-8f25-70ac8500a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.316560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1a3b4b3b-0615-4c56-8f25-70ac8500a3cd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:58.317804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a3b4b3b-0615-4c56-8f25-70ac8500a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:58.406516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 229/457] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:58.623966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0533-0196-40d6-a30b-5e6779d04734 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.624304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0533-0196-40d6-a30b-5e6779d04734 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.624420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0533-0196-40d6-a30b-5e6779d04734 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.626037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 229/458] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:58.631897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20999c8-7095-43f0-b270-4b26624d184f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.632187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20999c8-7095-43f0-b270-4b26624d184f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.632466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20999c8-7095-43f0-b270-4b26624d184f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.655230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e20999c8-7095-43f0-b270-4b26624d184f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:58.656470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e20999c8-7095-43f0-b270-4b26624d184f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:58.757532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 230/459] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:58.777479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e1c0-f389-406d-9363-8ca20fae49b8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.777760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e1c0-f389-406d-9363-8ca20fae49b8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.778080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e1c0-f389-406d-9363-8ca20fae49b8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.779667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 230/460] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:58.784509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f05efd-e1ec-47f0-8ddb-d642701a4f5a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.784799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f05efd-e1ec-47f0-8ddb-d642701a4f5a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.785075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f05efd-e1ec-47f0-8ddb-d642701a4f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.789245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e1c0-f389-406d-9363-8ca20fae49b8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:58.804323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d7f05efd-e1ec-47f0-8ddb-d642701a4f5a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:58.805519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d7f05efd-e1ec-47f0-8ddb-d642701a4f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:58.813746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:58.867297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:58.889339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 231/461] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:58.891143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c14d-42e8-49e5-b50c-489a565ffc8e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.891551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c14d-42e8-49e5-b50c-489a565ffc8e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.891848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c14d-42e8-49e5-b50c-489a565ffc8e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.893630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 232/462] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:58.899748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:58.900324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:58.900608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:58.941924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e1c0-f389-406d-9363-8ca20fae49b8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:58.943460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:58.946682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:59.005025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.005648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.005937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.006211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.027348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.029945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:46:59.056137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:59.057049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:46:59.057895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:46:59.057895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.057895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:59.058247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 231/463] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:59.064592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.075850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 233/464] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:46:59.076865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.177534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91833-cbe7-4f0c-9632-0ecb17b53ec2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.185886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:59.188031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:59.212397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.213134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.213519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.213894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.214725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:59.215123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.216630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.221473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e952b3-8b2e-4c8d-a6e8-5d88ea6c733f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.229555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 232/465] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:59.232261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.272823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.273603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.274027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.274400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.288355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.290017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:46:59.312569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:59.314128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:46:59.315020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:46:59.315020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.315020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:59.318839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 234/466] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:59.319790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.346441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801bb11-dcb8-45b7-a898-cca21a4c358a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.384047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.385043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.385611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.386163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.397847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.399629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:46:59.415508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:59.416589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:46:59.417601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:46:59.417601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.417601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:46:59.418980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 233/467] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:59.419892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.448136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:59.448872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52c36-3384-4946-9072-e7a6986082b7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.472778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:59.484625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.485233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.485515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.485786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.486460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.488163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.521934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 235/468] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:59 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:59.521934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.536651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42d1aa-a3fd-4fe1-b482-e828a75276d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.577913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.578555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.582007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.582712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.583864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:59.584250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.585790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.602223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 234/469] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:59.603790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.667313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256c7f-80d3-478b-9e32-6254b598e23b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.714308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:59.724753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.725779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.726522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.727103 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.736069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf63d-51cf-4eb7-a2d4-5249e4a3dc61 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.744585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.746761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:46:59.763189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:46:59.764134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:46:59.764979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:46:59.764979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:46:59.764979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:46:59.764979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 236/470] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:46:59.766163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.791852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.792466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.792743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.793018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.793684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:59.794064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.795460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.808916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:59.813692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 235/471] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:59.816480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.905635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f2435-8a03-430d-9ef2-157cce3251eb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.930915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b7c82-7e77-42b7-991f-64e3ea63dbed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:59.945484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.946255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.946604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.946954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.947675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.949195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.966070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:59.966619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.966910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.967179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.967884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:59.968133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:59.969494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf807ee-f4a5-4d0f-98ce-75b878732aea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:59.970955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 237/472] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:59 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:59.971723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.993129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 236/473] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:59.993129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554c2e-edea-4223-b6ee-56da9da8325e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:59.993129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554c2e-edea-4223-b6ee-56da9da8325e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:59.993129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b554c2e-edea-4223-b6ee-56da9da8325e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:59.993129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 237/474] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:46:59.993129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b554c2e-edea-4223-b6ee-56da9da8325e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:59.996781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:00.101893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:00.119066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1514f-4659-44b7-83d2-aa87714d3153 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:00.155268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b554c2e-edea-4223-b6ee-56da9da8325e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:00.165649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e799-dc3f-463a-82d1-af5180568bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:00.166290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e799-dc3f-463a-82d1-af5180568bc4 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:00.166571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e799-dc3f-463a-82d1-af5180568bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:00.166843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e799-dc3f-463a-82d1-af5180568bc4 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:00.167496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e799-dc3f-463a-82d1-af5180568bc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:00.168930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e799-dc3f-463a-82d1-af5180568bc4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:00.196815 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 238/475] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:59 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:00.197574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:00.199595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:00.200207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:00.200506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:00.200828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:00.201497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:00.202223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:00.202762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:00.203030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:00.224808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:00.226847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:00.230047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 238/476] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:46:59 2022] POST /identity/v3/users => generated 249 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:00.339128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 239/477] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:00.617953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:00.683561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:00.738134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c064e-3bf8-4f71-b169-d2f877f80bdb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:00.788309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:00.789063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:00.789448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:00.789826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:00.790632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:00.792094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:00.802302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 14:47:00.802302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 14:47:00.802302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [parameters: {'id': '43e9cc2dbdc44c41b9c1950164d02552', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 14:47:00.802302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 24 14:47:00.819535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return ref.to_dict() Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:47:00.820610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 14:47:00.821723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 14:47:00.822793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:47:00.823845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:47:00.824813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': '43e9cc2dbdc44c41b9c1950164d02552', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:00.825760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 14:47:00.827369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:00.829288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 239/478] 10.222.0.61 () {60 vars in 1131 bytes} [Sat Sep 24 14:47:00 2022] POST /identity/v3/roles => generated 144 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:00.833008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:00.925720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:00.962410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d27924-a562-4df1-8d62-fd2a090f6570 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:01.006005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e1b68-5528-40e1-85bd-a620a435480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:01.006936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e1b68-5528-40e1-85bd-a620a435480d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:01.007358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e1b68-5528-40e1-85bd-a620a435480d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:01.007768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e1b68-5528-40e1-85bd-a620a435480d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:01.023059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e1b68-5528-40e1-85bd-a620a435480d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-712e1b68-5528-40e1-85bd-a620a435480d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:01.028144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:01.029291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:47:01.030322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:01.032508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 240/479] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:01.038212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:01.184047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953465-d42d-4ebc-9848-c29f45d36d05 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:01.224155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:01.239901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:01.240750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:01.241176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:01.241593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:01.242425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:01.242889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:01.244395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9a83a-e16d-4b1a-8301-d8e796b99674 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:01.257671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 240/480] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:02.119566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2696-715b-42cf-9bb7-fd92bbbf6cfe None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.120570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2696-715b-42cf-9bb7-fd92bbbf6cfe None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.121095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2696-715b-42cf-9bb7-fd92bbbf6cfe None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:02.122955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 241/481] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:02.128143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.128954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.129450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:02.154702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:02.156209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:02.232136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 241/482] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:02.531856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2696-715b-42cf-9bb7-fd92bbbf6cfe None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:02.671670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2696-715b-42cf-9bb7-fd92bbbf6cfe None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:02.711803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:02.712441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.712724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.713002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:02.714850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:02.732286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:02.737105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:02.738290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:02.739089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 242/483] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:02.744286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:02.877369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9ce26-36ee-4efe-a050-633835bf8c69 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:02.902276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:02.914008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:02.914631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.914912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.915183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:02.915855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:02.916159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:02.922733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3e79-f2a5-47ce-9d0a-b83e880f56d4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:02.939690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 242/484] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:02.945369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:02.970407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bc2e-97df-45ce-b4d8-a59e7818ce9e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.970700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bc2e-97df-45ce-b4d8-a59e7818ce9e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.970969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bc2e-97df-45ce-b4d8-a59e7818ce9e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:02.972556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 243/485] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:47:02.974543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34d8ba-038b-4336-a2dc-80b56d243fed None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.974827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34d8ba-038b-4336-a2dc-80b56d243fed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.975102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34d8ba-038b-4336-a2dc-80b56d243fed None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:02.976521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 244/486] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:47:02.978890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a46b-66a2-46d0-829e-f9a9d5ca2309 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:02.979180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a46b-66a2-46d0-829e-f9a9d5ca2309 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:02.979457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a46b-66a2-46d0-829e-f9a9d5ca2309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.000980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cc42a46b-66a2-46d0-829e-f9a9d5ca2309 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:03.002240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc42a46b-66a2-46d0-829e-f9a9d5ca2309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:03.077018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 245/487] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:03.078304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f371f98-7c65-42b4-bde5-690ccc4c69cb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.078590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f371f98-7c65-42b4-bde5-690ccc4c69cb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.078866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f371f98-7c65-42b4-bde5-690ccc4c69cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.099272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9f371f98-7c65-42b4-bde5-690ccc4c69cb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:03.100623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f371f98-7c65-42b4-bde5-690ccc4c69cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:03.108735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e2090-39a8-483a-99e8-0aff0f2db7ac None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.148969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.150567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.151056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.151527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.163458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:03.165284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:03.181005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 246/488] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:03.182141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:03.183009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:03.183846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:03.183846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:03.183846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:03.184016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dabd2d-138d-4edc-b337-c54782ac2f32 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.184016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dabd2d-138d-4edc-b337-c54782ac2f32 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.184016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dabd2d-138d-4edc-b337-c54782ac2f32 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.184184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 247/489] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:03.188851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 243/490] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:03.190547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.191044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.191643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.194841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.211266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:03.212817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:03.252097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:03.282469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:03.324043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880e6a3-45c6-4f79-a98f-a4ba84dcf220 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.335489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 248/491] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:47:03.357851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.359044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.359462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.359895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.360664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:03.361146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:03.363008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:03.373924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.383763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 244/492] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:03.388372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.534338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f479ac-c447-4c32-b75a-a0074ce5188f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.565969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57692a-c814-4d6e-aa63-be0106d20e25 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.577033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:03.579768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.580893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.582086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.582754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.594118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:03.600099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:03.601376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:03.602387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:03.605002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 249/493] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:03.610687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.610818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.611399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.611681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.611957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.620946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:03.622892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:03.641247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:03.642395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:03.643312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 245/494] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:03.643894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.644507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:03.721893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5a156-236e-499f-aadf-a9139f1d331a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.748318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274a0e-167d-4a04-82ff-2dd0aa340154 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.757441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.758086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.758365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.758635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.768658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:03.773993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:03.774896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:03.775697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 250/495] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:03.776338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.790379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.790912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.791190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.791461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.792123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:03.792424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:03.793798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:03.802683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 246/496] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:03.803469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.835704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:03.877784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:03.890781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75e74c-cc5b-460c-98a8-4d2703966182 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.919333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9786eda-75fd-4047-b91b-07c385e8cfc5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:03.928920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.928920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.928920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.928920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.929238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:03.929238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:03.930047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:03.955255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:03.955886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:03.956172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:03.956450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:03.972229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 251/497] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:03.972395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:03.972495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:03.973441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:03.974402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:03.975215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 247/498] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:03.975856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.073894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.087928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46c2f-8cbc-40cb-badd-04f725104838 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.091669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbe4a0-3f3a-4817-bf2a-d0689d6c891b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.124455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.125086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.125373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.125651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.128907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.129509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.129788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.130137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.130853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:04.131161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.132530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.136385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.137845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.146553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 252/499] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:04.146553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:04.152899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:04.154259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:04.155106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:04.155106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.155106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:04.155738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.158062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 248/500] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:04.159493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.247191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd8b0-60f9-42ba-98d9-0d6ab9c6689d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.258537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357bbfd-da72-4cc4-a463-fbd66c5129d7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.279246 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527ea9d-c731-4b65-af26-cf38c79ee1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.279839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ea9d-c731-4b65-af26-cf38c79ee1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.280115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ea9d-c731-4b65-af26-cf38c79ee1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.280384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ea9d-c731-4b65-af26-cf38c79ee1f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.291172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.291755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.292035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.292306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.292951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:04.293253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.294657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.302760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 249/501] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:04.303369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.309595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.348476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527ea9d-c731-4b65-af26-cf38c79ee1f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.349985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527ea9d-c731-4b65-af26-cf38c79ee1f3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.358507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.407637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a4b4a-97d6-45e5-a4cd-da735e9eb063 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.441121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.441736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.442051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.442371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.447357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 253/502] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:04.448576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1019aab-54eb-4855-9a20-2542e6cefeba None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.448864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1019aab-54eb-4855-9a20-2542e6cefeba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.449136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1019aab-54eb-4855-9a20-2542e6cefeba None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.450539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 254/503] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:04.451060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1019aab-54eb-4855-9a20-2542e6cefeba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.456474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.457959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:04.472077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:04.473161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:04.474743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:04.474743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.474743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:04.474743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 250/504] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:04.474743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.528568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.553735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1019aab-54eb-4855-9a20-2542e6cefeba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.560850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.574187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2a2d5-55ba-4e4d-b8ef-f6103f470860 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.587238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.587833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.588109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.588374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.589004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:04.589301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.590676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.602154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 255/505] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:04.602875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.612599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.613456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.613569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.613763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.624801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.626279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1cd6b5b8-c959-47ea-b026-2435f2d50781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:04.640241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:04.641128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:04.642006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:04.642006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.642006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:04.642006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 251/506] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:04.642685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.642685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.642959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.660400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:04.661624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:04.701842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e8708-38f3-40e1-9015-f3ca77cdf64d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.728304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 252/507] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:04.729110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.735336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.735967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.736254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.736532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.747793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.749232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:04.764427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:04.765477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:04.766332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:04.766332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:04.766332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:04.766332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 256/508] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:04.766332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.782426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.788710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.853807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40ba86-2193-4338-b583-3aa2a4764ba9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.884950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a76906-874e-4fb4-a9ba-d7b8f2954dae None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:04.901655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.902488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.902765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.903035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.903666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:04.903969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.905326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.921239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 253/509] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:04.921239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:04.929809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:04.930500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:04.931094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:04.931372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:04.939802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:04.941255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:04.955029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:04.956017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:47:04.956900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:04.960760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 257/510] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:04.961583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.019971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.026583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfb5cd-fa2c-4c20-8539-1dbd7236f93f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.039737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.060152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.060755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.061030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.061303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.061948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:05.062253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.063424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928cfbd-271f-4fdf-82e3-c9d1f634627f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.063625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.074971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 254/511] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:05.074971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.100872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.101466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.101748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.102053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.102688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:05.102990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.104400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.117483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 258/512] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:05.118282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.174010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e9c5a-8fa6-459f-98fd-314ffafce298 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.211709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.212337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.212615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.212890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.214924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e17630-4e78-4dcf-8b69-5bed744e4bb7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.219857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.221314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:05.233203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:05.234159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 255/513] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:05.235046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.250125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.250726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.251010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.251285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.252009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:05.252319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.253750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.254877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.261816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.272598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 259/514] 10.222.0.61 () {56 vars in 1137 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:05.273393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.335756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc940bf-d842-45bf-9622-d07670967774 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.368623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.369170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.369451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.369723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.375139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5d3f-0ea9-4138-bec4-b91ffc418c60 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.380825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.382309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:05.396766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:05.397642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:05.398549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:05.398549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:05.398549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:05.398549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 256/515] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:05.398944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.410545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.411205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.411492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.411773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.418813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.420234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:05.432188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:05.433249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 260/516] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:05.434217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.466210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.488703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.499738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9a953-bfb4-49fa-8acd-e67af5f6152a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.532370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.532923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a169c-238c-47f4-bdec-f6031bda2aed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.533030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.533263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.533539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.540161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.541614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:05.553835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:05.554674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:05.555510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 257/517] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:05.555965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.573286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.573925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.574217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.574490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.575224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:05.575531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.576969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.593891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 261/518] 10.222.0.61 () {56 vars in 1139 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:05.593891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.659301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3a9c2-b2ff-4748-a156-4e2ca4fb85ff None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.685427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.693660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.694300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.694580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.694854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.697045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89858-3519-45a1-a08e-90a0f6569d4f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.710714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.732663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.733261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.733543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.733818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.734488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:05.734796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.736170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.747633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 262/519] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:05.748715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.762301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.764089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.847099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c580b3-13f9-40c0-99a0-b75a64491f6f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:05.860114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 258/520] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:05.860983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.889294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.890114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:05.890657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:05.890936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:05.891215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:05.892042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:05.894719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:05.896763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:05.914122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 263/521] 10.222.0.61 () {56 vars in 1137 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:05.914960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:05.948658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:05.969589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5bf7c-33b0-4d62-8a00-433fba8566d2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.002692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.003293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.003579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.003860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.004542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:06.005373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.006850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.016044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06888aa-3c00-4f36-bc14-6f0357b1833b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.024529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 259/522] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:06.025202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.050704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.051309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.052132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.052428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.109752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.121484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.122483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5857cf-eca7-401c-a842-3f43727aaab4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.122973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.151910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.156241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.156842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.157117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.157387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.158123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.159557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.181622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 260/523] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:47:06 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:06.182399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.252720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 264/524] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:06.254463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.297426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15bdbd-8cb5-4c45-8b33-57a04368888f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.332670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.333951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.334244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.334523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.348452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.351071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.355301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:06.370627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:06.371566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:06.372752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:06.372752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:06.372752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:06.372752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 261/525] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:06.373186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.377007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc997a-5c31-49e7-bfd7-6c5fd190f5a1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.417161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.424533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.425321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.425611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.426282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.481065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de7c70-85b2-44fa-bb3b-7d87fb098834 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.501543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.503025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.516958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.517559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.517838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.518139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.518858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=dc85d332eeaf4c649f51858fa608101f) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:06.519164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.521158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.526630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 262/526] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/role_assignments?user.id=dc85d332eeaf4c649f51858fa608101f&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:06.535210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.609517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.651358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 265/527] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:06.653067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.655699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d3375-2ed6-459a-a887-711573ef3f22 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.693368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.694154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.694498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.694835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.697511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.707872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.710057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:06.726353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:06.727387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:06.728248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:06.728248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:06.728248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:06.732098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 263/528] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:06.732908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.783137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14433e9-9413-4483-86c2-7096ca422961 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.823036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.824049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.825235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.825662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.826474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:06.826924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.828447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.842251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 266/529] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:06.843662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.863675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a195c7e-19b7-433c-9d37-d61ce5daa111 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:06.883332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.907121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:06.907760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:06.908042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:06.908312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:06.919366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:06.920884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:06.940769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:06.941683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:06.942615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:06.942615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:06.942615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:06.944708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 264/530] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:06.945805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:06.952352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:06.962541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba393-656c-4368-996c-cb8f216f9965 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.002274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.002988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.003276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.003551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.004198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.004507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.005969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.019419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 267/531] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:07.021012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.066926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108b403-b84f-4f5e-8cf8-3224d558a1ac None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.116033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.117246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.117784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.118477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.128602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.130242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.142275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4476e2-9ae2-4dcf-a578-1aabce51d0c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:07.142924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:07.143875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:07.145291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 265/532] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:07.145814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.181371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:07.188055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.188901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.189284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.189686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.190539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.190987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.192496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.206215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 268/533] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:07.207225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.224445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:07.281250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd807f-4d10-4ab0-b99b-b9b4a4432463 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.320016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9029f-e19b-47ee-9e17-73a9253320e2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.327141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.328428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.328856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.329270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.340479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.342074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:07.353654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:07.354714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:07.355755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:07.361863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 266/534] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:07.363138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.364964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.365781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.366432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.366854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.368604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.369085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.370690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.394661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 269/535] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:07.396732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.466894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:07.486628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487f94b-1e29-4d1b-9220-48118df48be3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.492274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:07.529253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361233e6-d602-452b-9c19-2d32c925c727 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.532521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.533231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.533626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.534031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.543400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.545722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:07.561915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:07.562861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 267/536] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:07.564440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.571835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.572449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.572732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.573006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.573703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.574458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.575901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.591129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 270/537] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:07.592011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.683769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005af241-056b-44fd-8c83-d7eadd534501 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.716508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6c6e3-6f75-489a-b9a3-343a8699cf5f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.717986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.718593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.718871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.719144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.719873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=09f49af0c95b458291e68167d99199f5) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.720181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.721691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.734239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 268/538] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/role_assignments?user.id=09f49af0c95b458291e68167d99199f5&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:07.735247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.750043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:07.771419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:07.783810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.784403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.784758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.785036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.785790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.786695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.788152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.819761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 271/539] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:07.830227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:07.947656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fd1b-72ba-4a80-92ac-1c395e98c458 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.988628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:07.989302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:07.989762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:07.990168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:07.991094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=84a66b2803614363a196ecd94828a3fa) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:07.991407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:07.992936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99207ff9-da81-4d76-9636-a32d0ff450f7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:07.995238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198051b7-1724-49fd-b8a9-14c05652c704 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:07.999710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 269/540] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/role_assignments?user.id=84a66b2803614363a196ecd94828a3fa&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:08.037641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:08.038700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:08.039296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:08.039757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:08.040693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=83758a542c2e4314871bf03fd2cca8b8) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:08.041219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:08.042914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265535e9-deae-46e8-ab50-2326bfe3c00d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:08.049648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 272/541] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:07 2022] GET /identity/v3/role_assignments?user.id=83758a542c2e4314871bf03fd2cca8b8&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:08.651927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881a132-2b8f-4888-a925-bd5626b19f6a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:08.652442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881a132-2b8f-4888-a925-bd5626b19f6a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:08.652562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881a132-2b8f-4888-a925-bd5626b19f6a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:08.654121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 270/542] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:08.659339 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3125-c8c2-4500-a238-287d75978e9d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:08.659750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3125-c8c2-4500-a238-287d75978e9d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:08.659894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3125-c8c2-4500-a238-287d75978e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:08.685054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e55b3125-c8c2-4500-a238-287d75978e9d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:08.686274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e55b3125-c8c2-4500-a238-287d75978e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:08.763493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:08.791406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 273/543] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:08.878076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e508a21-7606-4722-a252-4e28e3bcdef7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:08.878398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e508a21-7606-4722-a252-4e28e3bcdef7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:08.878634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e508a21-7606-4722-a252-4e28e3bcdef7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:08.880512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 271/544] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:08.887721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:08.888021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:08.888302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:08.919077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:08.920223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:09.011483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 274/545] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:09.111795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e508a21-7606-4722-a252-4e28e3bcdef7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.201358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:09.244088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e508a21-7606-4722-a252-4e28e3bcdef7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:09.292262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:09.292908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:09.293193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:09.293471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:09.307569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:09.312937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:09.313828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:09.314662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 272/546] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:09.315313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.320918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.475579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a7350-68ae-4c42-8509-f7223337c3f6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:09.496686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:09.509828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc96a6-d3e6-43a6-9313-c721091f765c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:09.530200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:09.530883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:09.531112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:09.531395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:09.544599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:09.550079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:09.550942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:09.551413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:09.551935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:09.552778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:09.553340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:09.555146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:09.556727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:09.561589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:09.562637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:09.563439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:09.568070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 275/547] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:09.568986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 273/548] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:09.569895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.574136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.673772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d85c-abb0-43b6-ab29-5340099b3f34 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:09.708924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:09.709984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:09.710498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:09.710968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:09.721475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:09.723305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:09.739032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:09.740450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:09.741467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:09.741467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:09.741467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:09.743488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 274/549] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:09.755601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.804281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77a271-0295-4d28-9645-95e21f2fa3f7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:09.845143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:09.862972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:09.863875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:09.864692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:09.864970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:09.865980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:09.866293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:09.867694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:09.877172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 276/550] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:09.883242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:09.940970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07235-f01c-4d00-a67f-c2d3c06c8de3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:09.982035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:09.982711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:09.982997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:09.983511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:09.984168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:09.984473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:09.985851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:09.998998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:10.001602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 275/551] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:10.007470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.120296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efcf4c-60f5-47d9-800f-e4cda74d2b6a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:10.128991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:10.161429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:10.162160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.162447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.162721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.170942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:10.172412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:10.185505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:10.187745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:10.189937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 276/552] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:10.191298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc2942-8866-45ff-b6e5-4459c49ec9dd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:10.199796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.278102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:10.279091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.279564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.280079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.295262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:10.297060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:10.313387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:10.314600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:10.315528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:10.315528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:10.315528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:10.318085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 277/553] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:10.330891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.344101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5ee9a-92b1-46eb-ae0d-2c8b93657049 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:10.384131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:10.385077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.385367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.385641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.386405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:10.389924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:10.389924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:10.431580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 277/554] 10.222.0.61 () {56 vars in 1135 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:10.437396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.465098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:10.526290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:10.542550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35965793-3aa8-493e-afd5-92d5de69bd2d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:10.566601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e3073-b09d-4e59-b8d5-25b6c2453a7f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:10.590813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:10.591986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.592728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.593633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.594799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:10.595628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:10.597931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9d9d1-b5c4-4fc5-b740-9c15d93b2227 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:10.603885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd741f-7cdc-4cab-9577-59e40fad2a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:10.604494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd741f-7cdc-4cab-9577-59e40fad2a95 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.604768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd741f-7cdc-4cab-9577-59e40fad2a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.605035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd741f-7cdc-4cab-9577-59e40fad2a95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.615005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 278/555] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:10.616495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cd6be-75dc-47fc-adf5-e7bc6fa42025 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.617356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cd6be-75dc-47fc-adf5-e7bc6fa42025 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.617859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cd6be-75dc-47fc-adf5-e7bc6fa42025 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.619986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 279/556] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:10.621433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79db915-ea92-4b65-a727-79e5b0de0d66 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.622342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79db915-ea92-4b65-a727-79e5b0de0d66 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.622973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79db915-ea92-4b65-a727-79e5b0de0d66 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.624643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 280/557] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:10.625619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79db915-ea92-4b65-a727-79e5b0de0d66 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.679017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd741f-7cdc-4cab-9577-59e40fad2a95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:10.681159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd741f-7cdc-4cab-9577-59e40fad2a95 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:10.713566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:10.751893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79db915-ea92-4b65-a727-79e5b0de0d66 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:10.789887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:10.790811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.791096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.791488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.800197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 278/558] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:47:10.800197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.800197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.800197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.802530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:10.803991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:10.816159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:10.817896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1f18f98b-44b3-4b22-a5e9-4f1515247dc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:10.818621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:10.819831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:10.820799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 281/559] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:10.821386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:10.821454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:10.822040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:10.840146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:10.841344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:10.844197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:10.890077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 279/560] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:10.890920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.912231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 282/561] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:10.913654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:10.957426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.009040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25534-721c-416e-8dd0-30ae288693c0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.043775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.044409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.044687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.044962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.055801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.057196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.061812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb917431-9d42-4112-94d9-282efe14acaa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0ed7e816-0a45-4b27-bd66-1d2e2af9251d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:11.072584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:11.073508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:11.074435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:11.074435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:11.074435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:11.078267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 280/562] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:11.079596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dd845-db9f-415e-8b0b-3fe06e96905a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.079882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dd845-db9f-415e-8b0b-3fe06e96905a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.080152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dd845-db9f-415e-8b0b-3fe06e96905a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.081770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 281/563] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:11.083064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dd845-db9f-415e-8b0b-3fe06e96905a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.108564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.109582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.110603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.111206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.112626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:11.113406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.116311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446d114-6e64-44cd-9584-4367f816569f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.131393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.149936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 283/564] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:47:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:11.149936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.149936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.149936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.171543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:11.172739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:11.199626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dd845-db9f-415e-8b0b-3fe06e96905a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.206484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.239845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.240453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.240734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.241009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.241644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:11.241741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 284/565] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:11.241977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.242609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.243468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.261190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 282/566] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:11.261190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.351564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd98d-6602-4bed-8b51-38aac7949c9e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.366132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561cb16-9d90-48f5-9ad5-c665c5998e62 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.381852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.390582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.391208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.391490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.391952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.400009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.400640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.400922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.401197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.402534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.404067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.412105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:11.417316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:11.418431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:11.419248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:11.425162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:11.426011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:11.428082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:11.428082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:11.428082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:11.428082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 285/567] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:11.428082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.428368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 283/568] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:11.428368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.451654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.527229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b6553-c73c-4e93-9ae9-8771656759ce None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.530170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ffff-63dd-4b54-b8f1-7ca80f1761ae None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.561238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.561867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.562179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.562458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.564030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.564832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.565409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.565691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.569758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.571241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.576375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:11.581632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:11.582667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:11.583719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:11.584650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:11.585629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:11.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:11.587381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 286/569] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:11.587981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 284/570] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:11.588453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.588793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.630824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.671750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.689110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d2087-f626-40e2-9b4a-ee7bb10cc825 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.689689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abd49e-26e7-4b29-958b-922933c1fd86 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.721254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.721855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.722055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.722183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.722427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.722645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.722924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.723065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:11.723195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.723928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.724358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:11.724660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.725308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.726018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.733554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 287/571] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:11.734354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.741959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 285/572] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:11.743152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.829951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e14c7-7f06-4d9b-8deb-b257f4c8888b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.831647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.840444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1174-9b26-4388-97a9-cd688379395e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.866145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.866763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.867042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.867338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.874715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:11.875310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:11.875591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:11.875863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:11.876515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:11.876821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:11.877215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:11.878219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:11.882594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:11.883641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:11.884586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 288/573] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:11.885442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.890057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 286/574] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:11.891133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:11.901852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:11.989999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2150a-655e-4bc2-9142-105901622a3e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:11.990201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b5555-faa9-48df-b598-40f63af59442 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.022510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.023100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.023198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.023717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.023827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.023990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.024080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.024265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.024979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:12.025288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.026741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.034919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.036910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.038756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 289/575] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:12.039587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:12.051447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:12.052606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:12.053644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:12.053644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:12.053644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:12.054163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 287/576] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:12.055228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.064038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.116375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.139886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20c472-3d21-49ed-85a1-e195138fec89 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.156067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753de1d-e189-4222-87b8-9e24c274040d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.173910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.174565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.174852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.175131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.182643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.184077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.189201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.189801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.190111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.190387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.191042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:12.191346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.192696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:12.195958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:12.196960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:12.197863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 290/577] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:12.198379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.209838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 288/578] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:12.209838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.279072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.297149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41065ba-be59-4ff7-b1d9-353e85f70173 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.305589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8ae62-e302-4665-a03f-4aea48e779cc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.327225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.330506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.331130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.331411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.331687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.339648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.340241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.340521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.340800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.341514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=756ac401fea141e68343168d6213d2b1) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:12.341822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.342661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.343293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.344084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.348969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 289/579] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/role_assignments?user.id=756ac401fea141e68343168d6213d2b1&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:12.349744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:12.358732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:12.359724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:12.360626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:12.360626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:12.360626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:12.360626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 291/580] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:12.361554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.471249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c172e9-bebe-449a-93ac-0c8e3e21c761 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.471591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0baa8-cdcf-4083-a31b-14277e43c72e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.509737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.510146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42747997-4673-479f-989d-d71139ee51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.510380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.510480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42747997-4673-479f-989d-d71139ee51ad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.510660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.510758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42747997-4673-479f-989d-d71139ee51ad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.510930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.511023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42747997-4673-479f-989d-d71139ee51ad None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.511573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:12.511679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42747997-4673-479f-989d-d71139ee51ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:12.511874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.511979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42747997-4673-479f-989d-d71139ee51ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.513274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.513372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42747997-4673-479f-989d-d71139ee51ad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.515814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.526521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 290/581] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:12.527356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.534432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 292/582] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:12.534432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42747997-4673-479f-989d-d71139ee51ad None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.574635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.630230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfbd4e-abb7-4aa9-a605-7bc31b7b7df6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.649728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42747997-4673-479f-989d-d71139ee51ad None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.664324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.665088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.665479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.665842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.676155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.677883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.687240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.687841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.688118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.688388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.689017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:12.689317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.690681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:12.693633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:12.694904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:12.695851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:12.695851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:12.695851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:12.695851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 291/583] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:12.695851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.712394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 293/584] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:12.712394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.768292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.796071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:12.801208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25c454-c51f-41a8-add0-e806cde8e32c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.818526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394a75a-e290-4bd8-aa63-48423ae46a89 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:12.835569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.836274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.836626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.836992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.844516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.846068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.852698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:12.853436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:12.853791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:12.854164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/dc85d332eeaf4c649f51858fa608101f/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:12.857518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:12.858611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:47:12.859682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:12.860440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 292/585] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:12.861313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.877404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=dc85d332eeaf4c649f51858fa608101f, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:12.878881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:12.901928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 294/586] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:12 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/dc85d332eeaf4c649f51858fa608101f/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:12.901928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:12.997232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dec0c-a7a9-4532-bd3b-7449a4b73c98 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.038886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8fac6-59b1-46e7-807e-d0a0a3c836be None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.041043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.041836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.043053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.043616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.044271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.045253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:13.045748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.047380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.070505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 293/587] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:13.071778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.081713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.082562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.083352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.083631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.092140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.093627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.095452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:13.110580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:13.111598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:13.113040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 295/588] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:13.113720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.192793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10debf-221e-452d-85aa-8ab12a928ba4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.229760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b864f-b767-46fb-8d07-4c486ee1dcaf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.232170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.233097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.233607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.234141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.235171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:13.235687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.237626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.258105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 294/589] 10.222.0.61 () {56 vars in 1137 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:13.259530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.267007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.267641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.267933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.268201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.276923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.278470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:13.291196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:13.292135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:47:13.293748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:13.296989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 296/590] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:13.297995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.325821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.353551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.380362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388a9e4-2fcb-4b9e-ac76-55235ba87647 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.408275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0394fa-3089-47ee-9801-03c92b97f0e9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.417215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.418054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.418428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.418801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.419733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:13.420164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.421865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.445990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 295/591] 10.222.0.61 () {56 vars in 1139 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:13.447097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.450821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.451749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.452276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.452774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.464815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.466662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:13.483207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:13.484156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:13.485042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:13.485042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:13.485042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:13.487643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 297/592] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:13.488687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.557381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4691bc8-b6b9-45ba-9300-3f37ae49d7d4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.573274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.597087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.598050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.598620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.599119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.599654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ed1fb-d506-40e3-bda5-9f0f9112194c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.600376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:13.600969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.603328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.605674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.630801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 296/593] 10.222.0.61 () {56 vars in 1137 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:13.632066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.640144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.640860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.641216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.641607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.654106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.655724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:13.671812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:13.672813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:13.673889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:13.673889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:13.673889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:13.673889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 298/594] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:13.678979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.746132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b65c28-7e7f-4b65-8753-cc386466853c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.783702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.784627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.785139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.785593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.786624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:13.787177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.788866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.792304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6e73d-8770-491e-8ba4-11ffb4e1445e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.802122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 297/595] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:13.803295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.821126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.829688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.830497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.830792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.831070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.844088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.846410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:13.863361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:13.865331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:13.866717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:13.866717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:13.866717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:13.869761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 299/596] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:13.870943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:13.871468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:13.932729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536233-8fc0-4b7d-950d-2d27244a9688 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.970899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:13.971864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:13.972379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:13.972907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:13.973935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:13.974487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:13.976185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:13.982332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b80a4-2240-46ba-83ca-b788ff374af2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:13.990050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 298/597] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:13.991245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.027101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.028356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.028878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.029360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.038432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.041030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:14.057852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:14.058886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:14.059809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:14.062696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 300/598] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:14.064442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.109927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff2306-a679-4897-b26d-3ffff37af3ae None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.120090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:14.133950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:14.160227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.161301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.161819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.162386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.163331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:14.163842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.165663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.179142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 299/599] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:14.180425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.188432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631240c-2c51-4a3a-8296-05c013c38806 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.224482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.225164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.225445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.225768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.233036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.234552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:14.246918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:14.247833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:14.248739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:14.252806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 301/600] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:14.253749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.300155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec6f85-2f5a-40e6-af6b-6e91b9ffe400 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.337789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.338876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.339454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.339953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.340843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:14.341367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.343293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.355132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:14.358274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c939c-6d8d-4918-85bd-fcbe39044ee9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.359331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 300/601] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:14.361005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.395693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.396336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.396652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.396931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.398078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:14.404983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.406407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:14.419446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:14.420435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:14.421322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:14.425510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 302/602] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:14.427568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.497244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783daa2c-4716-44d6-9723-10fd927b4c2c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.548169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.549140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.549774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.550476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.552015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:14.552628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.554428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.568219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 301/603] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:14.569399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.576490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4300ed6-1ba0-4eda-9758-b34fae8b7814 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.616411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1dbc3-ae7c-41bf-baef-9522916bc8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.617006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1dbc3-ae7c-41bf-baef-9522916bc8b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.617289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1dbc3-ae7c-41bf-baef-9522916bc8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.617567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1dbc3-ae7c-41bf-baef-9522916bc8b8 None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/09f49af0c95b458291e68167d99199f5/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.634809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1dbc3-ae7c-41bf-baef-9522916bc8b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=09f49af0c95b458291e68167d99199f5, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.636468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1dbc3-ae7c-41bf-baef-9522916bc8b8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.662000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 303/604] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:14 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/09f49af0c95b458291e68167d99199f5/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:14.663520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23730ecb-ada3-4ecf-b6ba-0ba086c46ac2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.663805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23730ecb-ada3-4ecf-b6ba-0ba086c46ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.664086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23730ecb-ada3-4ecf-b6ba-0ba086c46ac2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.665714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 304/605] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:14.666442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23730ecb-ada3-4ecf-b6ba-0ba086c46ac2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.704277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:14.704512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:14.721353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1e21-9da7-44fd-affb-5e5ba8b5b118 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.802378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.803078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.803334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.804962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.805717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:14.806056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.807541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4e430-2b05-402c-9389-5fa7d91e3266 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.837759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 302/606] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:14.842535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.842843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.843126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.869135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:14.870962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:14.883821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23730ecb-ada3-4ecf-b6ba-0ba086c46ac2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:14.935473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279191f6-6fbe-43fe-ab67-3f8a82c37749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:14.936148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279191f6-6fbe-43fe-ab67-3f8a82c37749 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:14.936433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279191f6-6fbe-43fe-ab67-3f8a82c37749 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:14.936707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279191f6-6fbe-43fe-ab67-3f8a82c37749 None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/84a66b2803614363a196ecd94828a3fa/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:14.956394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279191f6-6fbe-43fe-ab67-3f8a82c37749 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=84a66b2803614363a196ecd94828a3fa, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:14.958060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279191f6-6fbe-43fe-ab67-3f8a82c37749 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:14.970959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 303/607] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:14.972451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:14.974656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 305/608] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:14 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/84a66b2803614363a196ecd94828a3fa/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:15.083706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:15.112186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70d755-344e-41ca-8d7f-84c5c323e3b2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:15.163873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:15.164818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:15.165244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:15.165673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/83758a542c2e4314871bf03fd2cca8b8/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:15.184467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c078de-0862-4607-ab5d-a31b9d7b47de None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:15.184803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c078de-0862-4607-ab5d-a31b9d7b47de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:15.185299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c078de-0862-4607-ab5d-a31b9d7b47de None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:15.185958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=83758a542c2e4314871bf03fd2cca8b8, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:15.187686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 306/609] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:15.187996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:15.196701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:15.197069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:15.197272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:15.206133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 304/610] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:14 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/83758a542c2e4314871bf03fd2cca8b8/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:15.220533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:15.221794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:15.296905 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:15.311911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 307/611] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:47:15.432640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:15.542525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c167817-684e-4b79-ac75-45d1012e4daf None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:15.574288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:15.580749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:15.581440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:15.582122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:15.582406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:15.594524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:15.599883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:15.600839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:15.601702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:15.604063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 305/612] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:15.608669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:15.647440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:15.773170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540ab8f-64c6-4893-b6ac-f98c40c4ceb3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:15.791029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefdf40-5dca-4178-9e24-8c6c18a78cb9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:15.841640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:15.843095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:15.844140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:15.844763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:15.856630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:15.857444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:15.858713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:15.859143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:15.860362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:15.861736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:15.866682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:15.868903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:15.879053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:15.880308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:15.881308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:15.890436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 306/614] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:15.890679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 308/614] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:15.892197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:15.892630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:15.956512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:15.975494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:16.031673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f6f23-313f-493a-a59d-d9da08d96d2f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:16.050497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57a999-d64f-4a1a-aaf9-96169a5406f1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:16.071222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:16.072590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:16.073099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:16.073600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:16.085489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:16.087272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:16.103415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:16.104459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:16.105353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:16.105353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:16.105353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:16.113249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 309/615] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:16.115238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:16.149984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:16.150613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:16.150895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:16.151169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:16.151840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:16.152149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:16.153602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:16.172253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 307/616] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:16.178316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:16.230053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f61a0-58cb-40d8-b0ea-c0ab016b3407 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:16.232786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:16.271210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:16.272855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:16.273674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:16.274489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:16.275927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:16.276776 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:16.279634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:16.308227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 310/617] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:16.312461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:16.453650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69a891-2758-4b8c-80e7-6fff358c0609 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:16.496279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0c5fa-3743-4782-b96d-bcda76531b05 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:16.502262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:16.558846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:16.559121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:16.559804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:16.560277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:16.560748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:16.561459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:16.562301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:16.563036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:16.589955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:16.593058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:16.597813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:16.600469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:16.623049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:16.624909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:16.629543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:16.629543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:16.629543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:16.631013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 308/618] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:16.631327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:16.641949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:16.643856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:16.645707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:16.657777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 311/619] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:16.661930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:16.711603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:16.822757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd12209-aa6d-41ab-9448-24a834f01d55 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:16.858817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:16.859804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:16.860319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:16.860844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:16.861781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:16.862588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:16.864296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:16.887974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 312/620] 10.222.0.61 () {56 vars in 1135 bytes} [Sat Sep 24 14:47:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:16.889548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.003371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.007591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e36bf-0931-4423-ab0f-af697798b9e7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.025697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da9772-3aad-4023-b4f6-f006ebb1e0ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.076014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.076545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.076825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.077096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.077744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:17.078074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.079460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.092627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.093263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.093542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.093814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.105380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 309/621] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:17.106435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.108078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.110817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1f82b8a3-e7c7-478e-ae14-cce161f464bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:17.124482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:17.125639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:17.126730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:17.126730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:17.126730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:17.128385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 313/622] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:17.130262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc313ed9-7e06-4258-9bb2-5cba06be9ccc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.130692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc313ed9-7e06-4258-9bb2-5cba06be9ccc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.131140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc313ed9-7e06-4258-9bb2-5cba06be9ccc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.132440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 314/623] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:17.137211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc313ed9-7e06-4258-9bb2-5cba06be9ccc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.143984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.240022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cee341-6646-4bf7-9f39-dc9e6df12fd8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.260330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc313ed9-7e06-4258-9bb2-5cba06be9ccc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.286438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.288513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.289503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.290425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.299462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.300912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.303330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.304786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.305327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.305826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.306741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:17.307270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.309193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.311024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-39d4bc5f-87bf-42f3-8661-452ebb6e4383 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:17.313317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:17.314283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:17.315248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 310/624] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:17.317585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.317585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.317585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.329707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 315/625] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:17.329707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.335224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:17.336413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:17.417993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 311/626] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:17.419136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.454266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.464740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c153c-b592-4ac7-b3a4-a9f1d4e059dd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.527871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.528497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.528781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.529058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.529792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:17.530131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.532346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.574065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 316/627] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:17.576216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.612250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dee7e5-d5d9-48c7-b95d-2025eddeeb1a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.678542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.683306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.684152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.684575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.685001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.692292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.694038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5fc074ef-1d02-45a2-9f55-9876ddaf4fe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:17.705965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:17.707094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:17.708285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:17.710313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 312/628] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:17.711742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d4513-5329-4e20-9a2f-3fe531154990 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.712169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d4513-5329-4e20-9a2f-3fe531154990 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.712585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d4513-5329-4e20-9a2f-3fe531154990 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.718064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 313/629] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:17.719055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d4513-5329-4e20-9a2f-3fe531154990 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.732307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513de8-5b5c-4bff-a111-98d919f51be0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.774932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.776175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.776556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.777173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.803260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.830302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d4513-5329-4e20-9a2f-3fe531154990 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:17.858862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.860390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.863454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:17.864615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.864901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.865179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.865866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:17.866197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:17.867610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08792e-7e46-48a2-9720-0b4589c0885d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:17.885599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 314/630] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:17.886936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:17.887228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:17.887508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:17.906078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:17.907270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:17.914722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:17.979661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 317/631] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:17.980935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:17.998352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 315/632] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:17.999284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.077936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.102991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c050c-f70a-4251-92c9-d2bb04a41812 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.119593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc7663-25b8-47ee-8804-7b0b19a21094 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.139192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.139850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.140133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.140401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.151922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:18.155043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6415fa2-22cb-4539-bb9c-1c09b1675cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.155694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415fa2-22cb-4539-bb9c-1c09b1675cfb None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.155979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415fa2-22cb-4539-bb9c-1c09b1675cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.156254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415fa2-22cb-4539-bb9c-1c09b1675cfb None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/756ac401fea141e68343168d6213d2b1/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:18.156777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:18.157668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 318/633] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:18.159194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.175894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.178840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6415fa2-22cb-4539-bb9c-1c09b1675cfb None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=756ac401fea141e68343168d6213d2b1, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.180301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6415fa2-22cb-4539-bb9c-1c09b1675cfb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.194590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 316/634] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:17 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/756ac401fea141e68343168d6213d2b1/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:18.196073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd57b-b156-4e23-bddf-51aef1a83923 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.196460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd57b-b156-4e23-bddf-51aef1a83923 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.196809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abd57b-b156-4e23-bddf-51aef1a83923 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.198466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 317/635] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:18.203015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd57b-b156-4e23-bddf-51aef1a83923 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.285199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3364f-55b0-4202-8509-f855b0b639b2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.324860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.325534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.325820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.326129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.340415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.342557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.345154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abd57b-b156-4e23-bddf-51aef1a83923 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-dd20edca-ca1e-4ddf-9ec7-9017125c770c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:18.358763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:18.359827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:18.360711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:18.360711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:18.360711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:18.364959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 319/636] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:18.366255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.366398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.366549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.366825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.389794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:18.391020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:18.394151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.394974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.395876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.396156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.396856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:18.397134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.398582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.407351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 318/637] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:18.408403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.458372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.468564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 320/638] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:18.469222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.531757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b957b23-d2ae-41bb-aea3-81b5ffbc969f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.578138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b2550-8954-4a63-8d80-a9a1477969cf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.583272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.584260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.584700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.585127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.599436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:18.607397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:18.608485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:18.609512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:18.616841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 319/639] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:18.617918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.623037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.623203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.623482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.623849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.624613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:18.625044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.626577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.643616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 321/640] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:18.643616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.667380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.715915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.732476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab17a4d-324b-429f-bd53-90660183684f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.754999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807ca9-e4e8-457d-a70f-43dd8c95665e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.768335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.768955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.769229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.769498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.780628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.782111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.792872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.793483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.793764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.794074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.794740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:18.795048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.796440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:18.796603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:18.797480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:18.798422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:18.798422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:18.798422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:18.798422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 320/641] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:18.798653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.811024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 322/642] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:18.812696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.897063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.907061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b94be-c801-4673-ad0b-38b037c5abd1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.925207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dc004-4823-4a82-a11e-23d8bf5bf401 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:18.943686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.944310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.944586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.944848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.946560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:18.952072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:18.953517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:18.965141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:18.966138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 321/643] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:18.967082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:18.973078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:18.974320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:18.974607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:18.974879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:18.985757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:18.991338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:18.992281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 323/644] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:18.993202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.075392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec2eff-808c-4db7-b32d-2f53eb0ad270 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.102872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52c578-c3c1-48c0-aef4-f19045008610 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.112954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.113527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.113747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.114051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.114691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.114993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.116357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.133667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 322/645] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.133667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.145661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.151595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.152340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.152920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.153207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.166946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.168559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:19.183584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:19.184684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:19.185686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:19.185686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:19.185686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:19.185686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 324/646] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:19.189974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.221783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.237792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19596d3c-24db-4ed5-8cc5-d5b91870a8e6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.272466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.273069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.273346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.273614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.274319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.274624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.276045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.288213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 323/647] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.289024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.301662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928d1e-ef20-44c2-9029-53f723bb42d7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.339724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.340371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.340653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.340924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.341581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.341884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.343396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.356688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 325/648] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.356688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.360987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.393244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3458c-0ef5-4a95-b02c-8e5e93571bf5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.428743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.429328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.429895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.430207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.439408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.441468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.451733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:19.455901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:19.456841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 324/649] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:19.457840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.467608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f64037-8afe-497a-a561-0c364bda4d55 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.502185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.502901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.503080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.503326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.503954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.504253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.505648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.517337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 326/650] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.518147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.571444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a90600-08c7-4c6e-971c-c64e62236a33 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.600644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.609060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.609855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.610160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.610438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.611166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=dc85d332eeaf4c649f51858fa608101f) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.611475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.613164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.619177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 325/651] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/role_assignments?user.id=dc85d332eeaf4c649f51858fa608101f&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.619772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.643252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028efe1-3496-4f61-8efe-720994b2280c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.690431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.691437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.691920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.692883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.708327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.710815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:19.730496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:19.731631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:19.732692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:19.732692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:19.732692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:19.734530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 327/652] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:19.735583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84469487-342f-4353-96a1-de89fc3c5317 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.735743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.740999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.776727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.777515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.777919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.778308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.779170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.779613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.781346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.802529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 326/653] 10.222.0.61 () {56 vars in 1137 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.803668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.869225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da457715-6dad-4323-9f4e-2e6cdd2aa766 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.876963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:19.915717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.922605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.923094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.923526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.937837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.939342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.940094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6681e5-8372-44eb-a6b9-a0d127fccfd3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:19.952198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:19.953161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:19.954230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 328/654] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:19.954848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:19.979895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:19.980666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:19.981057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:19.981455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:19.982268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:19.982735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:19.984325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:19.998652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 327/655] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:19.999681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.028786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.071892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6647db6-7a26-4b2b-a7c8-f62948bc1182 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.115380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.116386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.117151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.117512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.122819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1f29-935d-4eba-827c-4b65ab2a902b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.137181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.161706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.162490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.162866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.163261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.164099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:20.164510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.166107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.190032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 328/656] 10.222.0.61 () {56 vars in 1139 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:20.191026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.212983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.215802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.285386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.328099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1658623-8081-4e9f-91f2-c288b051f68b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.339734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 329/657] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:20.339734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.379286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.380675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.381062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.381440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.389225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.391031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:20.407428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:20.408603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:47:20.409779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:20.412982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 329/658] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:20.413967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.448875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.461391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff66cb-4af9-4b3f-bc83-699915b57239 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.498269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.498880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.499162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.499437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.521142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45148d8-adef-4380-bcb1-db7ee9d8a91e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.533185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.556738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.557369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.557649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.558010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.570788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.572232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.576856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.578329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c3a3b777-d66d-4bbd-9339-f0d4afaa5078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:20.586777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:20.587753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:20.588660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:20.588660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:20.588660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:20.588660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 330/659] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:20.590258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bdf23-6c7a-4564-b16c-74bdea3f7755 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.590258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bdf23-6c7a-4564-b16c-74bdea3f7755 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.590258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bdf23-6c7a-4564-b16c-74bdea3f7755 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.591236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 331/660] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:20.591778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bdf23-6c7a-4564-b16c-74bdea3f7755 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.682407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.702806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 330/661] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:20.703766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bdf23-6c7a-4564-b16c-74bdea3f7755 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.704107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.743166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.743773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.744053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.744330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.745038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:20.745342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.746827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3785ba-bf6b-4e73-aaf5-3e3e38c1bf81 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.764043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 332/662] 10.222.0.61 () {56 vars in 1137 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:20.765138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.765419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.765687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.776157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.786484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:20.787683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:20.820230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb7be6-10d5-4a1d-994b-770a92e8c204 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.856840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:20.861967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:20.862195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 333/663] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:20.862296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:20.862296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:20.862727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.862806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:20.862806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:20.862806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:20.872183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 331/664] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:20.872978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:20.951381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:20.995326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3bcdb-bc8b-448c-8e80-7afb31dc1d7e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:20.999383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef52af2-dc04-4597-9eb4-e0ab436bab07 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.023184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.037202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.038016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.038369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.038712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.040762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.041757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.042134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.043362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.056784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.058794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:21.078894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:21.079866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:21.080759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:21.080759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:21.080759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:21.080759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 334/665] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:21.081501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.118753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.120369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.204856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15527d8a-e13e-4cd5-920e-ac77fe8f0526 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.226217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.237817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 332/666] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:21.237817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.242175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079539a-66cb-4104-9482-678593a97189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.242796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079539a-66cb-4104-9482-678593a97189 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.243070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079539a-66cb-4104-9482-678593a97189 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.243337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079539a-66cb-4104-9482-678593a97189 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.249935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079539a-66cb-4104-9482-678593a97189 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.251377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079539a-66cb-4104-9482-678593a97189 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-3079539a-66cb-4104-9482-678593a97189 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:21.263319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:21.264377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:21.265482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:21.266661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.271590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 335/667] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:21.271590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079539a-66cb-4104-9482-678593a97189 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.355877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ccd7-7e90-4825-8602-a9bd558bca13 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.378533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079539a-66cb-4104-9482-678593a97189 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.393434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.394064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.394342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.394614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.395275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:21.395593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.396911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.408148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 333/668] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:21.408936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.419659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.420285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.420587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.420856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.433439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:21.438630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:21.439547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:21.440379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 336/669] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:21.441078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.487083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.505880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.525216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783479e0-97ae-4aae-8d14-90e8ba96029f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.547755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443ca24-1b59-4d99-babc-fc99e04240a2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.561713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.562572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.563003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.563424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.575348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.577251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.581484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.582115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.582394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.583219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.583900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:21.584201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.585723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:21.593977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:21.595173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:21.596257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:21.596257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:21.596257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:21.598015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 334/670] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:21.601938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 337/671] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:21.601938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.603677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.711906 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143cae9-3203-4c12-bbeb-d03f0ba2f254 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.712087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b8833-d669-42e3-8b81-425dd774dc64 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.713443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.745890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.746570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.746782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.747070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.747684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:21.747983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.749347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.749569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.750222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.750629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.750907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.752933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:21.757831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 338/672] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:21.758638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.763529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.764966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:21.776957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:21.778108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 335/673] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:21.779172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.869464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30ede-5b6b-48b2-befb-51a1afdd735d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.895852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dbb51-344e-4ab6-b988-b41c4066d7b5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:21.906199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.906805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.907129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.907401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.908038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:21.908337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.909690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.925311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 339/674] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:21.926473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.944747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:21.945387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:21.945631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:21.945929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:21.946656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=09f49af0c95b458291e68167d99199f5) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:21.946964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:21.948377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:21.954223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 336/675] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/role_assignments?user.id=09f49af0c95b458291e68167d99199f5&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:21.955012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:21.965342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.038906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.070724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f7f46-505b-4468-a045-58e9291e66ca None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.095972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f43fe3-4803-46d2-aaf0-c0ec01be067e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.108003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.108758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.109145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.109428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.120848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.122405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.134854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.135561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.135913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.136255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.137022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:22.137398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:22.138637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:22.140702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:22.141797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:22.141797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:22.141797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:22.142036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2b948-6f35-42e8-9dd7-e5030c5685db None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.142123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 340/676] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:22.142123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.156111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 337/677] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:22.156111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d47816-9b6e-465f-aa61-4703774d9d04 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.156452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d47816-9b6e-465f-aa61-4703774d9d04 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.158143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d47816-9b6e-465f-aa61-4703774d9d04 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.158262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 338/678] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:22.158885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d47816-9b6e-465f-aa61-4703774d9d04 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.231028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.254814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea621479-e75b-45c8-8da9-93ef5c750535 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.275798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d47816-9b6e-465f-aa61-4703774d9d04 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.288013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.288651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.288938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.289413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.296124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.298418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.299904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-aa43b9c7-1976-4210-8d8b-45d37c7e339f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:22.312391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:22.313521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:22.314655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:22.319830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 341/679] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:22.319830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.319830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.320143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.320701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.321382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.321731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.322672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.322929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:22.323355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.324991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.339016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:22.340285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:22.342441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 339/680] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:22.343464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.426070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 342/681] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:22.427017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.451273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59d327-fa2f-45db-8153-a441349b0a38 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.492148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.492943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.493225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.493500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.494266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=84a66b2803614363a196ecd94828a3fa) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:22.494576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.495486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.496787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.505448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 340/682] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/role_assignments?user.id=84a66b2803614363a196ecd94828a3fa&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:22.506347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.562300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a57c6-b701-454d-9d5f-a6cda937294d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.563803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.606272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.606891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.607497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.607825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.608464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:22.609169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.611277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.637200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 343/683] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:22.643219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.665615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101646-fad1-43d0-b4fc-d77336b530e0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.706586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.707915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.708306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.708707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.719562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.721321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:22.737109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:22.738736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:22.739911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:22.741953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 341/684] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:22.742871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.806114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026e81-de73-4354-b4a8-c3816db54fdc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.836690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.847621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.848859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.849139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.849410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.850182 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=83758a542c2e4314871bf03fd2cca8b8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:22.850477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:22.851911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:22.865727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d5a14-8b70-41bf-a2ae-b175d681faf5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:22.867251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:22.869655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 344/685] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/role_assignments?user.id=83758a542c2e4314871bf03fd2cca8b8&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:22.870555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:22.904122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:22.904803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:22.905086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:22.905356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:22.921170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:22.929471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:22.930532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:22.931364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 342/686] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:22.938813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.016401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae86959-5524-49dc-bf1a-191df5bcdb7c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.056942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.057812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.058136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.058412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.059154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:23.059466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:23.061113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:23.084076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 345/687] 10.222.0.61 () {56 vars in 1135 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:23.089450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.158320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:23.165606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ffe1d-79d9-4475-90a9-b593a47bc181 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.213940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.215679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.216308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.216874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.218024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:23.218917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:23.221017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:23.224591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3732ae-ec45-4417-b694-76f81b3410fb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.232662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 343/688] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:23.238898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.266348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.267078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.267249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.267499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.275519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:23.359244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:23.361587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:23.384063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be508-ea53-48ed-9c07-9c0baf947500 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.458198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.458837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.459122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.459403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.462776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:23.486389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:23.490793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:23.517535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:23.518549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:23.519457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:23.519457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:23.519457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:23.524878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 344/689] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:23.529931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.571742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 346/690] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:23.578202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.702251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:23.714501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf786f03-a16c-4ea4-afcb-430327912a14 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.753160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f09eb-8b0c-4a89-b29c-f8c7539d848d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.784563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.786052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.786662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.787150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.793236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.794918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.795208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.795486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.796133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:23.796441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:23.797806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:23.813140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:23.814631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:23.827273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 345/691] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:23.829474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:23.834034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:23.839271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:23.840275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:23.841517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:23.841517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:23.841517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:23.841517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 347/692] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:23.851458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:23.951314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45a42-04cf-4bc5-a893-a97cbef7b4d5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:23.989178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:23.989812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:23.990136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:23.990486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:23.993528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:23.999998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.001811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:24.018929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:24.020523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:24.021561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 346/693] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:47:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:24.029037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.081615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769c92e-346b-4dbd-8079-843866b1ea41 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.159888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.160549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.161336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.161608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.162269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:24.162572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.163939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.191027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 348/694] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:24.197337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.224684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d380f9f-9fee-4dde-b676-c83095a6b14c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.279488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.280217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.280514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.280788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.281547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:24.281856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.283713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.310149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 347/695] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:24.313015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:24.314631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.392585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e0e27-8189-449e-92f7-184be2c84b3b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.405047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:24.432830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.433861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.434371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.434820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.442969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.444735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.452504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793c811-f5d6-4302-9c36-a9b7948e1bc4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-56f10dce-e3e9-45f1-bdb6-c00d53b494ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:24.457293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:24.458727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:24.459607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:24.462039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 349/696] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:24.463465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5fcad-8d24-48be-be97-109acc429e92 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.463943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5fcad-8d24-48be-be97-109acc429e92 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.464441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5fcad-8d24-48be-be97-109acc429e92 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.466158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 350/697] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:24.467503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5fcad-8d24-48be-be97-109acc429e92 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.494534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.495153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.495437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.496001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.569414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.571066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.580237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5fcad-8d24-48be-be97-109acc429e92 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.602634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:24.620999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.621796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.622171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.622514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.623373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:24.623755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.625256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1a738-5b4a-49d8-8826-926266cf8f55 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.633136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:24.646030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 351/698] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:24.646030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.646030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.646030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.666183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:24.667574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:24.688786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 348/699] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:47:24.689644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.746163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 352/700] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:24.747085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.798330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0c6fc-dea3-46a7-b09e-d12e9e371b32 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.834891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.835501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.835886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.836169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.836999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=756ac401fea141e68343168d6213d2b1) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:24.837312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.838780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.845309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 349/701] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/role_assignments?user.id=756ac401fea141e68343168d6213d2b1&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:24.854244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ca13-68f3-4eb5-a97f-10f6f2d9db3e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:24.867638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:24.895514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:24.896217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:24.896434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:24.896706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:24.908594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:24.910126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:24.927504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:24.928743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:24.929834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:24.929834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:24.929834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:24.933387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 353/702] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:24.935044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:24.961966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:25.086409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3217d-eab4-48ff-ad19-a49c2ea8379d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.138389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.139000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.139280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.139553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.140201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:25.140509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.141870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.151640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.157987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 350/703] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:25.162885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.263845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:25.295227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af880df0-1335-4981-b86c-74fed3a2d355 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.297065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bd06d-527c-44b2-a950-1fcbc1eca1e1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.335948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.337212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.337654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.338131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.340561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.341243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.341527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.342202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.346556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.348702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.354883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.356598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bb4e969f-a95e-4f85-a7cd-13d921b1fb1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:25.361618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:25.362966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:25.364564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:25.367584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 351/704] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:25.369488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2060be3-f9e9-4e50-93ba-2cc3e636614f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.370026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2060be3-f9e9-4e50-93ba-2cc3e636614f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.370489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2060be3-f9e9-4e50-93ba-2cc3e636614f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.372328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 352/705] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:25.377395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2060be3-f9e9-4e50-93ba-2cc3e636614f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-53e9c7e9-5792-47f1-b382-5e6f5d1572fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:25.377582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:25.378718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 354/706] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.379817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.404480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:25.406159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:25.422692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:25.502451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 355/707] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:25.503367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.505260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2060be3-f9e9-4e50-93ba-2cc3e636614f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.547604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.548464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.548923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.549351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.550242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:25.550738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.552793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecff5c-4ce6-4182-b492-1aa4ed81e843 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.563112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:25.568751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 353/708] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:25.569961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b113-44fe-47e4-a570-e85477842f52 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.570587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b113-44fe-47e4-a570-e85477842f52 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.570863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b113-44fe-47e4-a570-e85477842f52 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.572324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 354/709] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:25.574870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4b113-44fe-47e4-a570-e85477842f52 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.623993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59acb5-65e3-4cfb-ba78-60df37fc9a87 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.669354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.670017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.670368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.670994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.671753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:25.672144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.674586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8fd43-c791-461a-bc03-a6c55741e166 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.691950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 356/710] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:25.693547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.694566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.695049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.705720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:25.708083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4b113-44fe-47e4-a570-e85477842f52 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.721626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:25.724373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:25.745458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.746109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.746394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.746731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.747714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=dc85d332eeaf4c649f51858fa608101f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:25.748037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.749466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.755630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 355/711] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:25 2022] GET /identity/v3/role_assignments?user.id=dc85d332eeaf4c649f51858fa608101f&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:25.756416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.822448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 357/712] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:25.823858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:25.828332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:25.883621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590704a-241d-4db9-a61e-614f6122c816 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.945699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:25.946710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:25.947198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:25.947692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:25.948564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:25.951036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:25.961524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5c46-96a5-45ca-baed-6f2589bbc63f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:25.976068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 356/713] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:47:25 2022] POST /identity/v3/users => generated 254 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:26.020822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:26.022049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:26.022465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:26.022950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:26.023741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:26.025244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:26.032126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:26.049822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:26.051128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:26.051977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 358/714] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:25 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:26.057034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:26.183220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6dac8-ca2f-471a-9325-4be0bf6dd53a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:26.183410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:26.202165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:26.223844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:26.224458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:26.224743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:26.225019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:26.225657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] RBAC: Adding query filter params (name=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:26.225992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:26.227484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:26.236824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 357/715] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:26.238310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:26.324445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc608-776c-487b-a186-6d876f8d72d3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:26.359814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97646882-265d-4fd9-aebd-d03e5b8bb73a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:26.369210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:26.369856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:26.370186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:26.370463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:26.371055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:26.372487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:26.384120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:26.385141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:26.386013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 359/716] 10.222.0.61 () {56 vars in 1129 bytes} [Sat Sep 24 14:47:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:26.391921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:26.405324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:26.406309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:26.406788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:26.407256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:26.408076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:26.408594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:26.410253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:26.416562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 358/717] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:26.470836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:26.513370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c4397-6b0a-4680-9d01-e188bfcb2fec None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:26.564284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:26.565255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:26.565687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:26.566137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:26.567013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:26.567488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:26.569043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db63460-6643-4a85-80db-576dc071c39c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:26.576907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 360/718] 10.222.0.61 () {56 vars in 1124 bytes} [Sat Sep 24 14:47:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:26.582269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:26.656176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:26.713347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9922de-a8e8-4034-8f7a-7d3215673aa1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:26.748477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:26.749149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:26.749528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:26.749894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:26.750668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:26.751063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:26.752517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e899b-a94d-4856-8d2e-389724419981 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:26.758117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 359/719] 10.222.0.61 () {56 vars in 1124 bytes} [Sat Sep 24 14:47:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:27.641964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404649-578d-40e0-b410-fd834e182f8d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:27.642526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404649-578d-40e0-b410-fd834e182f8d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:27.642526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404649-578d-40e0-b410-fd834e182f8d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:27.644152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 361/720] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:27.649632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:27.650057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:27.650249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:27.678907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:27.680119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:27.764146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 360/721] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:28.196802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29404649-578d-40e0-b410-fd834e182f8d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:28.334619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29404649-578d-40e0-b410-fd834e182f8d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:28.372570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:28.386668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d31d6-33c7-4115-93ad-b42aa915ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:28.387331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d31d6-33c7-4115-93ad-b42aa915ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:28.387617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d31d6-33c7-4115-93ad-b42aa915ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:28.387897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d31d6-33c7-4115-93ad-b42aa915ffe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:28.402508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d31d6-33c7-4115-93ad-b42aa915ffe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d85d31d6-33c7-4115-93ad-b42aa915ffe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:28.407400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:28.408324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:28.409146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:28.411272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 362/722] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:28.417927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:28.485990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:28.503503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100195-b362-4f5c-8079-a80919f347b4 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:28.503806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100195-b362-4f5c-8079-a80919f347b4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:28.504068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100195-b362-4f5c-8079-a80919f347b4 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:28.506308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 363/723] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:28.511476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:28.511763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:28.512044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:28.545665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f896f3-0b51-4716-bb4b-09eae8f6a72e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:28.545860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:28.547089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:28.582073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:28.582842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:28.583120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:28.583390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:28.584174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:28.584480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:28.585848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:28.595079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 361/724] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:28.601231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:28.692963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 364/725] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:47:28.750653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4f1a5-b51a-4e80-b033-9bc01e6e8b9d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:28.752252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:28.785009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:28.785758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:28.786076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:28.786358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:28.797197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:28.798739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:28.823431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:28.824399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:28.825290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:28.825290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:28.825290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:28.825290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 362/726] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:28.831343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:28.967188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d414a3-40b7-47d8-9515-e45b9696ac01 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:28.995535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.013848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.014413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.014689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.014959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.015599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:29.015900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.017267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e36fc2-a1e9-4541-836a-12f6369fcb1b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.032465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 365/727] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:29.038230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.069426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90366-70f7-457a-aecc-ef13a70f5b83 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.069717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90366-70f7-457a-aecc-ef13a70f5b83 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.070020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90366-70f7-457a-aecc-ef13a70f5b83 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.071484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 366/728] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:29.077943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.078521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.079005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.101821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:29.103101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:29.169796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e5032-3742-4775-a9d5-6877b5fda340 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.174991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 367/729] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:47:29.183076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.205339 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.206009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.206292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.206566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.215269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.216707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-82feb1a9-73e8-421f-ae8d-1624111f34b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:29.228039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:29.228980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:29.230355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:29.233775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 363/730] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:29.235031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0632be-5c90-4e7c-ab41-9fb74e5f5ac9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.235329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0632be-5c90-4e7c-ab41-9fb74e5f5ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.235613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0632be-5c90-4e7c-ab41-9fb74e5f5ac9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.237098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 364/731] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:29.242273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0632be-5c90-4e7c-ab41-9fb74e5f5ac9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.259081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.315652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebc5f6-39bc-41f1-b35a-e297e1ce9af3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.324451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.354124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319aa50-f44a-4d24-844d-30c310e880b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.354689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319aa50-f44a-4d24-844d-30c310e880b4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.354969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319aa50-f44a-4d24-844d-30c310e880b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.355247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319aa50-f44a-4d24-844d-30c310e880b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.368388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319aa50-f44a-4d24-844d-30c310e880b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-2319aa50-f44a-4d24-844d-30c310e880b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:29.373246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:29.374199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:29.375071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:29.378225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 368/732] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:29.380202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.380491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.380764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.384057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0632be-5c90-4e7c-ab41-9fb74e5f5ac9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.400211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:29.401725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:29.428640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.429949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.430892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.431689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.432955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:29.434121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.436322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.461294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 365/733] 10.222.0.61 () {56 vars in 1135 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:29.462362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.480147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 369/734] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:29.483188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.539181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.571201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca43f8c-0025-46d0-b024-7856674c946d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.606610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.609046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.609754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.610171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.610546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.611311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:29.611739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.613460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.613771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6390-defb-44a2-8b07-8760f2b7947d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.627454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 366/735] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:29.627454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.650181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3c055-f344-4543-bfc8-476895c4493a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.651342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3c055-f344-4543-bfc8-476895c4493a None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.651729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3c055-f344-4543-bfc8-476895c4493a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.652131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3c055-f344-4543-bfc8-476895c4493a None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.652880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3c055-f344-4543-bfc8-476895c4493a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.654399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3c055-f344-4543-bfc8-476895c4493a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.671901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 370/736] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:47:29 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:29.673242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072176d5-f583-45fe-8dcc-4423481d6680 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.673533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072176d5-f583-45fe-8dcc-4423481d6680 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.673818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072176d5-f583-45fe-8dcc-4423481d6680 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.678063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 371/737] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:29.678063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072176d5-f583-45fe-8dcc-4423481d6680 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:29.749506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52243170-5114-47c8-91e8-1310ad23ca1b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.794264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.808434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.809059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.809436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.809691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.819703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072176d5-f583-45fe-8dcc-4423481d6680 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:29.857458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:29.858047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.858330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.858606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.870481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.871986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e8b05170-4ca9-4808-aff3-ef298ad8ca3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:29.888520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:29.889501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:29.890475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:29.890475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:29.890475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:29.890644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:29.890716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 372/738] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:29.892810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:29.892943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:29.893035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:29.893035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:29.911678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:29.912861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:29.914926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:29.990505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 373/739] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:29 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:47:29.991821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.013260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 367/740] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:47:30.014916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.052726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.123976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ff8b-e110-485e-bf1e-eabfbed8e8c7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.130379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe679c68-801a-420a-b89b-1a680e93ee1f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.166560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.167515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.168034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.168499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.169331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:30.169827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.170451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.171088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.171373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.171653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.172288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.183761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.189488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:30.190002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 368/741] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:30.191085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:30.194499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:30.195431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:30.196257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 374/742] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:30.196934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.299587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf4b9a-ea29-4684-bc3e-939d0a543883 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.306309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.317716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672e167-a3fc-4fb3-b052-0d3c275623fe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.334920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.335546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.335832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.336109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.343110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.344661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.356353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.356983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:30.357086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:30.363129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:30.364189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:30.364713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.364713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.364837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 375/743] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:30.364837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.376750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.378372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:30.393699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:30.394685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:30.396335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:30.396335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:30.396335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:30.396335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 369/744] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:30.396335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.432740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.478105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851bf38-69e0-4ac0-8538-8d7b038a35c9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.500252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea1aac-4b6c-428d-ab6c-584464f63fc0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.513926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.514904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.515189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.515607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.516268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:30.516570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.517941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.531232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 376/745] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:30.531232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.541202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.541863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.542193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.542474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.543144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.544511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.562513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 370/746] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:47:30 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:30.563865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.569260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.661879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6779a9c-026d-410b-90e7-335e1b2f08f5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.697108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.720377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41864a24-7747-4fde-b28c-88df51764d22 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.721083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.721935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.722855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.724412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.725411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:30.725925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.727614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.755202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 377/747] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:30.756781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.759836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.760744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.761750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.761750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.762225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:30.762628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.764088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.783186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 371/748] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:30.783186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.858465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:30.883022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c3684-a42f-4ea9-be51-941404184254 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.918594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.919448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.919571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b191-e859-4871-95b8-b7bb6d9c1dec None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:30.919969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.920383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.936718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.936718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:30.953305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:30.954375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:30.955267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:30.955267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:30.955267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:30.956331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 378/749] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:30.961223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.963426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:30.963654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:30.963883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:30.964160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:30.975068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:30.977267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:30.994728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:30.995711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:30.996583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:30.996583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:30.996583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:30.996583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 372/750] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:30.997336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:30.998205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.085846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23579f3-3a73-4b70-a498-e07e1ef42623 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.118996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b9f0a-beb4-46bf-a4f2-c85297e4740b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.123877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.124515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.124793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.125060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.131843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.134450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.136072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:31.150058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:31.150973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:31.151859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 379/751] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:31.152342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.164909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.165849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.166473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.166941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.167934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:31.168429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.170237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.190594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 373/752] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:31.191139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.263357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.273958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15382b-ddb6-4bef-bc71-68f85e2205d8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.310618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.311597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.312116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.312584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.313417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:31.313941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.315606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.326600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9e54a-b453-4e94-aa16-0ed01da3fbef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.330157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 380/753] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:31.331309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.374167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.375220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.375722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.376192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.377075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:31.377573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.379361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.393991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 374/754] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:31.395366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.432405 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.474606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d9b75-2a7a-4d45-a09a-e8e8fab9f6ac None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.521950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a93ee7-3ff4-419a-a6c0-b095251c0331 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.527427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.528082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.528367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.528645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.540709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.542954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:31.556840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:31.557808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:31.558756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:31.565614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 381/755] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:31.567121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.569410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.570081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.570384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.570847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.581083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.581755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.584614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:31.601351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:31.603360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:31.605981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:31.609990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 375/756] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:31.611361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.681747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d229117-164d-4285-a7f4-f2b8e66e66fe None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.719078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.719837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.720295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.720568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.721415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=756ac401fea141e68343168d6213d2b1) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:31.721781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.724050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.737933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 382/757] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/role_assignments?user.id=756ac401fea141e68343168d6213d2b1&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:31.737933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.740831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.753602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92134-3c41-4b45-94f8-67108f9c3c92 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:31.800851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:31.801879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:31.802421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:31.802899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:31.803873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:31.804388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:31.806114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:31.829843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 376/758] 10.222.0.61 () {56 vars in 1149 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:31.843268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:31.873928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:31.930363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cf7ce-2f2c-4c92-8a08-12b6aa00c231 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.013554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.014206 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.014477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.014752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.015460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:32.015769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.017520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a0c50-ec50-485e-b5e2-2e85d870273e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:32.036763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 383/759] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:32.038506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff0ba-9316-4da1-b3f9-e3b4f5cf7fe8 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.038807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff0ba-9316-4da1-b3f9-e3b4f5cf7fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.039232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff0ba-9316-4da1-b3f9-e3b4f5cf7fe8 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.042037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 384/760] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:32.042833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff0ba-9316-4da1-b3f9-e3b4f5cf7fe8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:32.051950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850e0fa-47af-4661-9af8-007d252e87e3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.099336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.099956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.100239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.100514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.123002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:32.179907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.181494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:32.191467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff0ba-9316-4da1-b3f9-e3b4f5cf7fe8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.230542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ca4c1-a4a5-42b7-ac58-e7edf81550dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.231208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ca4c1-a4a5-42b7-ac58-e7edf81550dd None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.231489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ca4c1-a4a5-42b7-ac58-e7edf81550dd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.231763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ca4c1-a4a5-42b7-ac58-e7edf81550dd None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/dc85d332eeaf4c649f51858fa608101f/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.233701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ca4c1-a4a5-42b7-ac58-e7edf81550dd None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=dc85d332eeaf4c649f51858fa608101f, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.235186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ca4c1-a4a5-42b7-ac58-e7edf81550dd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:32.237880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:32.254020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 385/761] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:32 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/dc85d332eeaf4c649f51858fa608101f/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:32.255708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.256476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.256761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.286537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:32.287792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:32.312130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 377/762] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:47:32.318189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:32.379513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:32.387736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 386/763] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:32.440058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd241bbb-8ebf-4c78-8615-ea9f3f178fcc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.480965 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.482038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.482522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.483044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.494906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.496727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:32.513271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:32.514317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:32.515209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:32.515209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:32.515209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:32.517400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 378/764] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:32.524013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:32.650722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1c3cf-bc1e-4726-9347-380259070680 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.674857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:32.686487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.687095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.687369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.687637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.688274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:32.688571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.689941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:32.702509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 387/765] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:32.708372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:32.726177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:32.887260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bae46e-8ffc-4e33-9286-5930d054e32c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.890415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:32.893551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4cfc9-4236-4019-ade8-0284ba6a162e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:32.945535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.947054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.947549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.948037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] PATH_INFO: `/v3/services/network` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.948805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.950504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:32.954063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:32.954727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:32.955024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:32.955317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:32.961778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:32.962865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:32.963768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:32.967529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 388/766] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:32 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:32.975303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:32.976856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:32.978392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:33.000328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:33.001260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:33.002166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:33.004614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 379/767] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:33.007859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:33.075608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:33.136415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdab3-9f02-4b32-9fef-b0ab1eddd2c3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:33.147179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145bde-3f23-4a2f-acec-87307289c987 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:33.180949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:33.181827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.182147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.182426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.183426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:33.184086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:33.185987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dded89-06d8-4cf1-a657-9e7b8d807ae3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.189268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:33.189895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.190299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.190581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.191210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] RBAC: Adding query filter params (name=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:33.191537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:33.193441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.200547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 380/768] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:33.201783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199526ea-91d5-4014-b11f-9d2223c41848 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.204379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 389/769] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:33.205254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:33.206734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199526ea-91d5-4014-b11f-9d2223c41848 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.207031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199526ea-91d5-4014-b11f-9d2223c41848 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.209931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 381/770] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:33.210799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199526ea-91d5-4014-b11f-9d2223c41848 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:33.238096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:33.319019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6249fb-9591-4f3f-be7c-c7f7fce3ba14 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:33.330288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199526ea-91d5-4014-b11f-9d2223c41848 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:33.332386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:33.353836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:33.354477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.354756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.355035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.355848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] RBAC: Adding query filter params (type=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:33.355958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:33.357374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51599bcf-e61d-4c12-8fdd-510fa40d3313 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.365983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 390/771] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:33.365983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.365983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.365983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.377378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:33.378249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.378629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.379028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.380545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=82bd05cad09d4763a79fc7400921c3b0) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:33.381042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:33.382715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.386356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:33.387740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:33.389509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 382/772] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:33 2022] GET /identity/v3/role_assignments?user.id=82bd05cad09d4763a79fc7400921c3b0&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:33.394600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:33.549600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:33.569094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff786cbf-7f30-4445-a444-eecb257fd7cd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:33.591672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 391/773] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:33.607223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:33.608239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.608723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.609191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.610146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] RBAC: Adding query filter params (interface=public, service_id=7c61eb358d944e4cb50edc3efeae5e62, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:33.610659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:33.612305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.621331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 383/774] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:47:33 2022] GET /identity/v3/endpoints?service_id=7c61eb358d944e4cb50edc3efeae5e62&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:33.626647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:33.731848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f605-7dec-4fe2-9072-df0e3a918180 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:33.740699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:33.768852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:33.769466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:33.769805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:33.770613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:33.771220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:33.772716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:33.778351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 392/775] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:47:33 2022] GET /identity/v3/services => generated 772 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:33.917650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:34.026595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458e0e1-2b15-4e7b-ac7a-86342d76c8ce None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:34.066425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:34.067144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.067426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.067702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.068336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:34.069779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3d1c376e-15c2-48ec-b527-4d5eab19c520 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:34.081782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:34.083588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:34.084425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:34.085252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 384/776] 10.222.0.61 () {56 vars in 1129 bytes} [Sat Sep 24 14:47:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:34.093600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:34.156166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d50f-0f87-4270-9c7c-34c08792c330 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.156673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d50f-0f87-4270-9c7c-34c08792c330 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.156750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932d50f-0f87-4270-9c7c-34c08792c330 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.158416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 385/777] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:34.166350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.166742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.167460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.192471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:34.194178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:34.204895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:34.214524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773369d-aa02-424a-a7e0-02b334155383 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:34.254857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:34.255742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.256171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.256604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.257375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:34.257976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:34.259625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:34.265049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 393/778] 10.222.0.61 () {56 vars in 1124 bytes} [Sat Sep 24 14:47:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:34.272717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:34.290214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 386/779] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:47:34.318430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:34.386223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41fc6d-a61c-4191-8e7a-8aaa035593f1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:34.432981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:34.433601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.433881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.434177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.434806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:34.435111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:34.436474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:34.446341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 394/780] 10.222.0.61 () {56 vars in 1124 bytes} [Sat Sep 24 14:47:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:34.451887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:34.633553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966df38c-1606-4598-8077-d6980927b2c3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:34.643702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:34.681307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:34.681931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.682217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.682488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.683190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] RBAC: Adding query filter params (interface=public, service_id=7f1ea97a209144c38efcb4665794150c, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:34.683497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:34.685959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:34.697619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 387/781] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:47:34 2022] GET /identity/v3/endpoints?service_id=7f1ea97a209144c38efcb4665794150c&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:34.706859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:34.720490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:34.824206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ab161-f725-4fef-a699-32a302d2c9f3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:34.844212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:34.849830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9603cde-0edd-44e0-aa88-a25edc86be14 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:34.869094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325ee2b-5500-43f3-83cf-a3aec62195cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:34.869094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325ee2b-5500-43f3-83cf-a3aec62195cb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.869094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325ee2b-5500-43f3-83cf-a3aec62195cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.869094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325ee2b-5500-43f3-83cf-a3aec62195cb None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.869532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325ee2b-5500-43f3-83cf-a3aec62195cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:34.871105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325ee2b-5500-43f3-83cf-a3aec62195cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:34.877130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 388/782] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:47:34 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:34.878649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7d0a2-ef96-4f12-b0ce-387170968683 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.879668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7d0a2-ef96-4f12-b0ce-387170968683 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.880146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7d0a2-ef96-4f12-b0ce-387170968683 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.881840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 389/783] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:34.888547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.889622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.889896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.902413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:34.903052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:34.903333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:34.903604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:34.916160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:34.917819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:34.929088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:34.935611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:34.936714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:34.937562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:34.939806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 395/784] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:34.946489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.058606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:35.061127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 390/785] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:35.116205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5bbc9-baa4-4041-8bf6-194b12faef2e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:35.151749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:35.152361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:35.152635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:35.152904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:35.153566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:35.153934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:35.155461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:35.165380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 396/786] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:35.171013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.279236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b861-e5ed-4e68-baac-0ed51c613e17 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:35.314590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:35.315140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:35.315416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:35.315733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:35.326624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:35.328055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:35.342675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:35.343624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:35.344514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:35.344514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:35.344514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:35.344514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 391/787] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:35.349696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.375825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:35.469145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4ab1-c6b8-4278-890b-bc5b1a8fd712 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:35.482140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.533628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:35.534273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:35.534556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:35.534831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:35.536291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:35.537203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:35.541074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:35.565680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 397/788] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:35.573374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.588536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:35.671522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:35.706365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf05ccf-b46c-4e44-b537-e61a1f81380e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:35.718989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd55f19-d520-4f2b-aca6-441d6b367ae3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:35.766584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:35.767221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:35.767498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:35.767768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:35.784061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:35.785558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:35.793415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:35.794489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:35.794937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:35.795381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:35.807347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:35.808276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:35.809216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:35.811464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 398/789] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:35.816335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.830415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:35.837676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:35.839846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:35.841877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 392/790] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:35.845645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:35.995534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f013df9-9bb4-4802-923e-5e7feef8f576 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.025872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157230-1271-421b-b6b2-436f0597251e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.055822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.056451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.056737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.057016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.057770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:36.058108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.059626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.066430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.086277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.092027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 399/791] 10.222.0.61 () {56 vars in 1135 bytes} [Sat Sep 24 14:47:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:36.097240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.111429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.112315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.113249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.113734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.114625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:36.115003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.116447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.136381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 393/792] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:36.143473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.251197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb27f5-83f3-4f62-98d5-86e85bdf79a3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.286686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.287338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.287578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.287852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.294169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909698a5-52ef-4f9d-a03c-970379f99fd1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.298266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.299721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f784f1c8-43be-41ec-bad6-80794cf9bef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:36.314593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:36.315555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:36.317152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:36.317152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:36.317152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:36.317152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 400/793] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:36.317399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880da06-1199-4020-9a59-e8b36d5f12d5 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.317668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880da06-1199-4020-9a59-e8b36d5f12d5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.317849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880da06-1199-4020-9a59-e8b36d5f12d5 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.323153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 401/794] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:36.323912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880da06-1199-4020-9a59-e8b36d5f12d5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.337126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.338198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.338701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.339207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.352046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.353968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0210e1a7-2724-4fed-b779-d37593185b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:36.370389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:36.371331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:36.372258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:36.372258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:36.372258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:36.374514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 394/795] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:36.376063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.376553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.377103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.381106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.400313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:36.402015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:36.424778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.434068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880da06-1199-4020-9a59-e8b36d5f12d5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.471363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.471926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.472213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.472481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.473390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:36.473577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.474974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.481487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 395/796] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:36.482753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.502514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 402/797] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:36.508578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.676383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea0542-67cf-4ac7-9470-45122e681904 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.696320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496cb30-63f3-4c3f-8a17-1bb7bcb315db None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.708983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.713615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.714231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.714510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.714781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.715415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:36.715710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.717079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1f48-c73f-4471-873f-ddfc36b9df5a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.728607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 396/798] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:36.729847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8388-b8cd-45b1-9025-c3a8b17bd3d9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.730151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8388-b8cd-45b1-9025-c3a8b17bd3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.730447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8388-b8cd-45b1-9025-c3a8b17bd3d9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.734415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 397/799] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:36.736917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8388-b8cd-45b1-9025-c3a8b17bd3d9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.739608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.740240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.740522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.740797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.748064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.749512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.758693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1631148d-f081-45b7-9411-aeed8917c13b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:36.763146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:36.764031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:36.764915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:36.770333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 403/800] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:36.771727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.772019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.772408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.791453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:36.792758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:36.850268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8388-b8cd-45b1-9025-c3a8b17bd3d9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:36.860978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 404/801] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:36.861952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.895912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:36.896795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:36.896956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:36.897358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:36.905890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:36.906789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:36.919259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:36.920250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 398/802] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:36.921178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:36.993530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:36.998327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfa25c-475e-4302-8dda-ec6c91e053ab None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.004664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.030279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f1e64-134f-45d5-a0a4-0a3e8e2bf991 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.037972 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.038824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.039253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.039687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.040605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:37.041069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.042814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.061458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 405/803] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:37.062678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.067160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.067914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.068299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.068681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.079536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:37.085522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:37.086692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:37.087620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:37.093088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 399/804] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:37.094194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.175166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f5bf-326e-4838-a36b-27489323ca66 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.196779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c33f41-ce4b-4143-a455-9493496ed7e0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.216625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.217331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.217687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.218074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.218811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:37.219123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.220567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.233412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.234098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.234495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.234775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/756ac401fea141e68343168d6213d2b1/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.236668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=756ac401fea141e68343168d6213d2b1, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.238153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 406/805] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:37.239022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.242850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.246456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.259695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.260346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 400/806] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:37 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/756ac401fea141e68343168d6213d2b1/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:37.261119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.382900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd858685-e832-43c8-a2d0-8cd86bea3c0e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.400526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf408ab2-c645-469f-a082-a3d25ee09713 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.423316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.424176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.424577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.424928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.425601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.427823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:37.437697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:37.438811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:37.439957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 401/807] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:37.440761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.453246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.454197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.454629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.455067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.455900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:37.456366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.458082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.469974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 407/808] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:37.471134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.568399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.594193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede018-78e0-42f2-aebd-2b47da18451c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.625472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.639082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.639704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.639989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.640264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.640902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] RBAC: Adding query filter params (name=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:37.641209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.642637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.647781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 408/809] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:37.648565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.675826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e4434-daf6-4976-bf59-9c3908c53e15 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.713273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.713883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.714102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.714377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.762579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390d5c9-f973-4108-8911-3870dcb35a1c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.781949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.783411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.798780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.799412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.799695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.799968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.811264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.812810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.815438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-05291a66-01e1-4af7-9fb2-f90211c1024c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:37.829516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:37.831093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:37.831964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:37.831964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:37.831964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:37.831964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 409/810] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:37.832963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dea7c-d683-42be-8112-e7e882a98f96 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.833275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dea7c-d683-42be-8112-e7e882a98f96 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.833529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dea7c-d683-42be-8112-e7e882a98f96 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.839564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 410/811] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:37.839564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dea7c-d683-42be-8112-e7e882a98f96 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.854054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:37.905120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 402/812] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:47:37.905939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:37.950265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dea7c-d683-42be-8112-e7e882a98f96 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:37.984054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:37.985062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.985649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.986157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:37.986865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] RBAC: Adding query filter params (type=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:37.987232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:37.988963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760288c4-0ddc-4e97-9e96-dacf2119af78 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:37.998411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 411/813] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:37.998411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:37.998411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:37.998411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.014024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:38.015202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:38.024427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab13d0-9e5d-4abc-8284-40a8d30b6ea7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.059863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.060760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.061043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.061321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.061992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:38.062356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.063682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.075142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:38.076300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 403/814] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:38.077381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.090723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 412/815] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:38.091328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.114330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:38.201918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb05ebf-83f8-4bde-9e5b-1ce5cc89283a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.230769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e09426-362d-4d3c-b818-bec125953e27 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.239489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.240157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.240557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.240935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.241701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.243273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.267877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 413/816] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:47:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:38.269467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.304340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.305094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.305383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.305715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.329649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.331163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:38.360571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:38.361589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:38.363861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:38.363861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:38.363861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:38.363861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 404/817] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:38.366158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:38.374960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.467830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:38.512621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe6583-f200-4a66-9c4d-7b1347dea969 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.567092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.567305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67704c07-41c9-4c39-8390-c3dcdf6b4a32 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.567750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.568029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.568300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.569889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:38.570240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.571621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.594831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 405/818] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:38.595039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.611622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.612256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.612843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.613388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.620748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.622547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:38.635440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:38.636406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 414/819] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:38.637998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.710412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5d8c-a8df-4832-93b0-daac9ed630a7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.736289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:38.744688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.745420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.745697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.746005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.746583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.748150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.751340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663fe4d-c883-469c-8560-d2edaf6a36f8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:38.757390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:38.758482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:38.759933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 406/820] 10.222.0.61 () {56 vars in 1129 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:38.761114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:38.761190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.795508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.796162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.796434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.796888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.803636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.805249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:38.817399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:38.818337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:38.819261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 415/821] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:38.819787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.872864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5484fd1-7713-46c9-a0ad-91f7a0da88d4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.913977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.914582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.914864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.915138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.915865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:38.916167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.917592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.935216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6c0a-a0f9-4471-9c37-b0e826233bde None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:38.945288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 407/822] 10.222.0.61 () {56 vars in 1149 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:38.945288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:38.972020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:38.972673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:38.972957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:38.973224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:38.973852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:38.974191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:38.975554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:38.980681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 416/823] 10.222.0.61 () {56 vars in 1124 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:38.981388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.002474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.012936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.058516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bf3ec-cb66-441c-b550-223672f252a3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.089623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d64b7-7711-452b-ad4b-fd3680994cc8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.093824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.094479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.094759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.095026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.095709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:39.096011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.097421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.114303 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 408/824] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:39.114303 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.129137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772794d0-084c-47df-affe-87128d146859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.129819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772794d0-084c-47df-affe-87128d146859 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.130130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772794d0-084c-47df-affe-87128d146859 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.130402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772794d0-084c-47df-affe-87128d146859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.141671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772794d0-084c-47df-affe-87128d146859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.143144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772794d0-084c-47df-affe-87128d146859 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-772794d0-084c-47df-affe-87128d146859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:39.159072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:39.161442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:39.162444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:39.162444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:39.162444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:39.162444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 417/825] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:39.162444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772794d0-084c-47df-affe-87128d146859 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.233120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cc034-d0bc-4ecd-ad8b-551ae587b70f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.245656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.248194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.267019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772794d0-084c-47df-affe-87128d146859 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.272738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.273347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.273628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.273910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.274597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:39.274837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.276181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.281318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 409/826] 10.222.0.61 () {56 vars in 1124 bytes} [Sat Sep 24 14:47:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:39.286919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.302339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.302948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.303227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.303502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.304219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=dc85d332eeaf4c649f51858fa608101f) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:39.304530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.306055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.311840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 418/827] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:39 2022] GET /identity/v3/role_assignments?user.id=dc85d332eeaf4c649f51858fa608101f&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:39.314759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.411285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cceba-2cb2-4bef-a135-dc5b9c1de398 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.450737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.451368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.451652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.451926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.452593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:39.452900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.454366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.469683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973ca67-e804-4842-8cb5-22e7f710c1b0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.473399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 410/828] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:39.482018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.513505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345bc29-d8f7-4356-8353-9c22ad4112f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.514516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345bc29-d8f7-4356-8353-9c22ad4112f7 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.514939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345bc29-d8f7-4356-8353-9c22ad4112f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.515489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345bc29-d8f7-4356-8353-9c22ad4112f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.516649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345bc29-d8f7-4356-8353-9c22ad4112f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.518032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345bc29-d8f7-4356-8353-9c22ad4112f7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.526963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.538981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 419/829] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:47:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:39.540520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf4186-d1e9-47c8-be9e-96825d319f0b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.540897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf4186-d1e9-47c8-be9e-96825d319f0b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.541178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf4186-d1e9-47c8-be9e-96825d319f0b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.545942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 420/830] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:39.548991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.549276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.549546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.556128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.570984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:39.571416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:39.621861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e027d28-c973-446f-a430-5647139d9739 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.667476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.668114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.668390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.668658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.670369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 421/831] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:47:39.680799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.682307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:39.695057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:39.696012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:39.696900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 411/832] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:39.702326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.823872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c6476-e547-4959-8b6a-b8fada0cc9bc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:39.854744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.860915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:39.861803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:39.862360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:39.863081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:39.864567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:39.865087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:39.866756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:39.880211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 422/833] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:39.885841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.960241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:39.983291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:39.994089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3943b4-113c-43f4-8b68-2e9430dae1c1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:40.028635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:40.029298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:40.029584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:40.029867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/82bd05cad09d4763a79fc7400921c3b0/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:40.044564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=82bd05cad09d4763a79fc7400921c3b0, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:40.046073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:40.061839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 412/834] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:39 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/82bd05cad09d4763a79fc7400921c3b0/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:40.062592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436a44-ac07-409f-84d0-e7e79372b19b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:40.102625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:40.103192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:40.103466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:40.103734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:40.116268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:40.123706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:40.124716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:40.125548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 423/835] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:40.132093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:40.248381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bde51-bbeb-4a26-8a72-0b99dc8f4b7e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:40.291438 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:40.292051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:40.292334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:40.292608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:40.293285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:40.293593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:40.294995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:40.306592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 413/836] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:40.312196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:40.371022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:40.420962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667d608-516a-41ae-a270-b0eadb220325 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:40.455811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:40.456500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:40.456783 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:40.457200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:40.468953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:40.470423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:40.485318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:40.486345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:40.487252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:40.487252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:40.487252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:40.487252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 424/837] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:40.494416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:40.498087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:40.611665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa384d70-f5c2-4e5b-8b68-ef44a0dcf293 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:40.647310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:40.647843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:40.648125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:40.648407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:40.649107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:40.649409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:40.650849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:40.663969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 414/838] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:40.669378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:40.777702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216524-bd0d-40a1-b2d2-d3435f7270ef None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:40.781294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:40.813567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:40.814219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:40.814497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:40.814765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:40.822916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:40.824429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:40.835708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:40.836632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:40.837568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 425/839] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:47:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:40.844540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:40.915840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:40.962964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef9ad2-e5ed-4e79-a13a-533d4da360b7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:41.000288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:41.001287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.001774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.002365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.003299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:41.003881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:41.005570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:41.023881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 415/840] 10.222.0.61 () {56 vars in 1135 bytes} [Sat Sep 24 14:47:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:41.029790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:41.137105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fed4c-d366-4c91-88cb-53f80eb751fd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:41.170900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:41.171439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.171716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.171985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.202858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:41.241109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:41.242587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:41.343085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 426/841] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:47:41.349232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:41.456163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f48f9-4d63-4a6a-ac32-7a7180981761 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:41.478001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:41.495694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:41.497126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.497638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.498204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.509842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:41.511578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7fae6257-f847-4bac-adce-95cf5fd367b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:41.526838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:41.527787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:41.528670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:41.528670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:41.528670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:41.530844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 416/842] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:41.537118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:41.603571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:41.642753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd7128-a4b5-4191-91d5-03af097d06f0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:41.670707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041f967-b734-43dc-b5cb-f89429041e3f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.671363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041f967-b734-43dc-b5cb-f89429041e3f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.671650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041f967-b734-43dc-b5cb-f89429041e3f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.673130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 417/843] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:47:41.680090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92cf4-62b0-4bea-85cc-71675dfdd41f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.680372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92cf4-62b0-4bea-85cc-71675dfdd41f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.680647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92cf4-62b0-4bea-85cc-71675dfdd41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.684609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:41.685457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.685967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.686495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.687391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:41.687896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:41.689467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:41.698995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ac92cf4-62b0-4bea-85cc-71675dfdd41f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:41.700271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ac92cf4-62b0-4bea-85cc-71675dfdd41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:41.702652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 427/844] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:41.710032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:41.788826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 418/845] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:41.832852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d22e83-9de9-4522-845b-84a84cb58c4e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:41.839499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170932dc-a893-4be7-963c-142204fa32c6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.839787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170932dc-a893-4be7-963c-142204fa32c6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.840055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170932dc-a893-4be7-963c-142204fa32c6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.841624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 419/846] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:47:41.846401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.846682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.846956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.860374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:41.866448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:41.867740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:41.869840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:41.870857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:41.871414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:41.871842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:41.879943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:41.881669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:41.894034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:41.895658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:41.896551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:41.898682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 428/847] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:41.904344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:41.958321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 420/848] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:47:42.018044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e8dd-1215-41b2-956a-5823c860ae6c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.053080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.053688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.054007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.054287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.054976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:42.055286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.056726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.067307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.070216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 429/849] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:42.078863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.099103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:42.133778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:42.173287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a4260-77e6-45b5-8fab-b33432b6b11a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.190399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a45b4-f94d-4776-8b1c-8166d23fb792 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.213005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.213637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.213946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.214219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.214801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.216160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:42.225262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:42.226248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:42.227136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 421/850] 10.222.0.61 () {56 vars in 1132 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:42.239506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.240804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.241693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.242458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.242935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.244740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=756ac401fea141e68343168d6213d2b1) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:42.245255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.247012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.261395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 430/851] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/role_assignments?user.id=756ac401fea141e68343168d6213d2b1&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:42.265753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.373550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:42.388647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c33b-3240-46c7-9a06-cfc2e5e00e30 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.403269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2221c-7af6-44a9-940b-ef0dd3bc10e3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.414969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:42.436955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.437585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.437892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.438193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.438828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:42.439132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.440494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919285-3b59-4b29-a945-9e5f90c3916b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.448353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 422/852] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:42.449325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.450317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.450781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.451373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.451962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.452642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580969-a603-407f-af75-5887589d952f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.452937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580969-a603-407f-af75-5887589d952f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.453216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5580969-a603-407f-af75-5887589d952f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.453641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.454704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 423/853] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:42.455410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5580969-a603-407f-af75-5887589d952f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f97d0c4d-0264-480b-b76e-8f4f3542f12e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:42.463510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:42.464510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:42.465357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 431/854] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:42.466205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.466205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.467336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.485327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:42.486858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:42.564888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 432/855] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:42.570806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.575986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5580969-a603-407f-af75-5887589d952f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.616304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.617194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.617580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.617988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.619119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:42.619259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.620797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.630336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 424/856] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:42.695259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02112f2-5f3a-4004-938a-4fcbf43228cd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.697237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:42.733401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.734054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.734335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.734613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.735239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:42.735542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.736920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.742692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 433/857] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:42.748631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.782767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:42.839946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:42.869801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc95cd7-6c47-47ed-8e8f-cb536bbe2ae6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:42.907281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:42.908237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:42.908817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:42.909284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:42.910194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:42.910670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:42.912272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:42.917996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 425/858] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:42.973460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658b5ba-dcc6-4252-80df-9fce770f0761 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:43.013531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:43.014209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:43.014493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:43.014766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:43.027059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:43.032079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:43.033046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:43.033864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 434/859] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:47:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:43.039796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:43.143109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:43.155404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ad90f-25da-46ed-89dc-5ad1fac5af68 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:43.190473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:43.191002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:43.191304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:43.191579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:43.192237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:43.192537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:43.193973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:43.202993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 426/860] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:47:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:43.208951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:43.239411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:43.314375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b0e62-cdcb-4ecf-891a-df748d41af30 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:43.353648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:43.354307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:43.354589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:43.354860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:43.366310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:43.367754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:43.382465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:43.383417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:43.384302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:43.384302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:43.384302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:43.384302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 435/861] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:43.391178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:43.497635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0753-f9b3-4cf8-b794-b8c97a0a4f83 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:43.531629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:43.532230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:43.532511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:43.532773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:43.533410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:43.533711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:43.535152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:43.548400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 427/862] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:43.554042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:43.638400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:43.662801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4ba7f-45d1-4ebf-9c39-3d396758f0a7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:43.696999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:43.697980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:43.698588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:43.699112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:43.707639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:43.709348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:43.720985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:43.721940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:43.723534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:43.726002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 436/863] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:47:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:43.731928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:43.746691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:43.833692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f725e-07df-4265-8664-311392202e81 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:43.867841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:43.868447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:43.868722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:43.868990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:43.869710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:43.870047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:43.871466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:43.889645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 428/864] 10.222.0.61 () {56 vars in 1149 bytes} [Sat Sep 24 14:47:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:43.895839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:44.010100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968d70-8658-4354-84eb-1de87424403c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:44.044632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:44.045246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.045528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.045802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.054725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:44.115270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:44.116705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:44.222488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 437/865] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:47:44.227809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:44.294294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:44.332919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd2c57-e02b-4238-9724-38c14c21860d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:44.369079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:44.369650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.369960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.370244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.383195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:44.384692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-206f79d6-5526-4eec-a979-7999a60e1904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:44.401802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:44.402998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:44.403885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:44.403885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:44.403885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:44.403885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 429/866] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:44.411068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:44.451354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:44.490604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5c1aa-28f1-4e92-a81c-823a2ca130cf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.490915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5c1aa-28f1-4e92-a81c-823a2ca130cf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.491162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5c1aa-28f1-4e92-a81c-823a2ca130cf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.492770 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 430/867] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:47:44.499067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.499353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.499628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.519483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:44.520706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:44.523198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c809c-457c-4d66-ad8f-ad900cce9624 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:44.563523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:44.564135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.564394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.564666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.565412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:44.565724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:44.567849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:44.580735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 438/868] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:44.586861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:44.602011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 431/869] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:44.692411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:44.696408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5485f2d-b9f9-45b8-9038-75eaef391306 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:44.731865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:44.732495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.732777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.733048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.740922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:44.742418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:44.753821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:44.754755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:44.755669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 439/870] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:44.762364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:44.815237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:44.868498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be13fd-177c-4b32-bd8a-14e56b6932cb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:44.902996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:44.908152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:44.909154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:44.909645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:44.910732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:44.911650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:44.912166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:44.913836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:44.930644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 432/871] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:44.936801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:45.036292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a2f8-96fe-431d-b285-496a2513b222 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:45.048240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3017-b9d5-40c4-94b9-a05d5b93673f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:45.059731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:45.073364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:45.074074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.074356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.074630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.075202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:45.076553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:45.085581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ec21f058-d408-48e5-98c6-a7751a71c8f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:45.085794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:45.086862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:45.088312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:45.089052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.089052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.089052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.089269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 440/872] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:45.089340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=82bd05cad09d4763a79fc7400921c3b0) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:45.089340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:45.089460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277029-5b1e-46d6-a4a4-368a9a78d6e2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.089460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277029-5b1e-46d6-a4a4-368a9a78d6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.089460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277029-5b1e-46d6-a4a4-368a9a78d6e2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.089647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15fe9d-7fa2-47e0-b8ac-80f748859dc8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:45.094272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 441/873] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:45.095026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277029-5b1e-46d6-a4a4-368a9a78d6e2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:45.095562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 433/874] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:44 2022] GET /identity/v3/role_assignments?user.id=82bd05cad09d4763a79fc7400921c3b0&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:45.103466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.104116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.104596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.132273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:45.132615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:45.133883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:45.229843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 434/875] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:45.240006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277029-5b1e-46d6-a4a4-368a9a78d6e2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:45.279265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:45.280216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.280724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.281211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.281849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:45.282179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:45.283547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71535f57-afc0-4bbd-a01b-10b2a453bfb7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:45.288684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 442/876] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:45.293866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:45.346549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037cda-9c46-4a7a-a1dd-1a33cdfdd46e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.346862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037cda-9c46-4a7a-a1dd-1a33cdfdd46e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.347100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037cda-9c46-4a7a-a1dd-1a33cdfdd46e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.348759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 443/877] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:45.353447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.353729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.354035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.374877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:45.377423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:45.413949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9981ef-e922-41fb-9903-14b6b5cbf70f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:45.419169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:45.458864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:45.459482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.459771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.460084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.461311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:45.461684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:45.463361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:45.465463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 444/878] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:45.473005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 435/879] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:45.500625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:45.519231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:45.610967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237472a6-e0f7-43d6-8c39-bb81a4259b90 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:45.649310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd693071-938b-4753-bf62-5c002e585917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:45.649949 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd693071-938b-4753-bf62-5c002e585917 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.650231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd693071-938b-4753-bf62-5c002e585917 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.650502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd693071-938b-4753-bf62-5c002e585917 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.651157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd693071-938b-4753-bf62-5c002e585917 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:45.652532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd693071-938b-4753-bf62-5c002e585917 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:45.669861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 445/880] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:47:45 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:45.771154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:45.884036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfb41a-e4d9-44bd-bd21-b9e2c40761ef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:45.929540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954de9f4-4b74-4414-abe0-c27f6368d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:45.930191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954de9f4-4b74-4414-abe0-c27f6368d2a8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:45.930475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954de9f4-4b74-4414-abe0-c27f6368d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:45.930749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954de9f4-4b74-4414-abe0-c27f6368d2a8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:45.931410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954de9f4-4b74-4414-abe0-c27f6368d2a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:45.938358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954de9f4-4b74-4414-abe0-c27f6368d2a8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:45.950410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 436/881] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:47:45 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:47.430428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7bc22-5d54-480e-a76d-e61655ad77d8 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.431001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7bc22-5d54-480e-a76d-e61655ad77d8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.431001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7bc22-5d54-480e-a76d-e61655ad77d8 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.432599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 446/882] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:47.438049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d82368-50aa-4631-b8c7-f14fa2c7ddb2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.438900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d82368-50aa-4631-b8c7-f14fa2c7ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.439389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d82368-50aa-4631-b8c7-f14fa2c7ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.458438 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-30d82368-50aa-4631-b8c7-f14fa2c7ddb2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:47.459955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-30d82368-50aa-4631-b8c7-f14fa2c7ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:47.478211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:47.542026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 437/883] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:47.680380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc410c-dac9-4422-b1f7-3300fa28bf22 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.680999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc410c-dac9-4422-b1f7-3300fa28bf22 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.681433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc410c-dac9-4422-b1f7-3300fa28bf22 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.683217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 447/884] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:47.693221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e28ddb-868a-4895-aa16-69b90a8cf9f8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.693761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e28ddb-868a-4895-aa16-69b90a8cf9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.694278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e28ddb-868a-4895-aa16-69b90a8cf9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.713548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-50e28ddb-868a-4895-aa16-69b90a8cf9f8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:47.715144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-50e28ddb-868a-4895-aa16-69b90a8cf9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:47.790082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 438/885] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:47.815581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc410c-dac9-4422-b1f7-3300fa28bf22 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:47.896978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:47.913529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9273c0-b008-4c2b-89ab-50b8f23910fb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.913846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9273c0-b008-4c2b-89ab-50b8f23910fb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.914119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9273c0-b008-4c2b-89ab-50b8f23910fb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.915521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 439/886] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:47.920894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.921241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.921524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.935425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc410c-dac9-4422-b1f7-3300fa28bf22 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:47.939806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:47.941833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:47.974733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:47.975368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:47.975655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:47.976001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:47.988042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:47.993151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:47.994192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:47:47.995097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:47.999265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 448/887] 10.222.0.61 () {56 vars in 1120 bytes} [Sat Sep 24 14:47:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:48.000194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.020269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 440/888] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:48.068023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.082067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.138176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69705441-877e-4d41-b378-c62f0bbce831 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.183527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.184415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.184703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.184966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.186161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:48.186426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.188065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856edbe3-a3e5-4a85-9a47-4cb52b318cce None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.188599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51c69f-b1b1-4f1f-a788-04faac97643a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.190435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.205092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 449/889] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:47:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:48.205092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ee28-726f-46f9-b647-449babb1de36 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.205092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ee28-726f-46f9-b647-449babb1de36 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.205092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ee28-726f-46f9-b647-449babb1de36 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.208700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 450/890] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:48.209690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490ee28-726f-46f9-b647-449babb1de36 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.230284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257ce5d-2d3e-4a21-8d3b-dc547c21bd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.231182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257ce5d-2d3e-4a21-8d3b-dc547c21bd49 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.231475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257ce5d-2d3e-4a21-8d3b-dc547c21bd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.231754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257ce5d-2d3e-4a21-8d3b-dc547c21bd49 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.232415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257ce5d-2d3e-4a21-8d3b-dc547c21bd49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.234268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257ce5d-2d3e-4a21-8d3b-dc547c21bd49 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.250479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 441/891] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:47:48 2022] POST /identity/v3/services => generated 241 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:47:48.263440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.263798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.264116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.287492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:48.288718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:48.344663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490ee28-726f-46f9-b647-449babb1de36 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.362467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.369948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 442/892] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:48.371097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.393890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.395259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.395538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.395812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.412707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.415152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:48.434672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:48.436122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:48.437062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:48.437062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:48.437062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:48.437062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 451/893] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:48.444518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.497768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04151a33-a076-4d78-ad8f-a5f41b80985b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.511711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.538257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.538902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.539189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.539456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.540040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.541865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:48.552840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:48.553828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:48.555335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 443/894] 10.222.0.61 () {56 vars in 1132 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:48.559840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.573671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82888d9c-4163-429d-b20a-db9296cefb12 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.623939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.625035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.625670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.626068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.626393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.627309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:48.627619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.629261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.644218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 452/895] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:48.645107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.667605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d5cd5-bf6c-4f97-bf32-70d5adb1897d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.703180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.703710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.703994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.704292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.704904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:48.705211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.706607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.711644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 444/896] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:48.712232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.754340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4505b4-f126-4ec3-a8bd-c9c3bb2735ad None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.761619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.790015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.790649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.790927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.791197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.791771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.793135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:48.807187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:48.810213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:48.811288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 453/897] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:48.812261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.825499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f723131-5cda-4097-94ac-561130053fc3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.854648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:48.866894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.867754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.868191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.868609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.877148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.878930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:48.891453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:48.897998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 445/898] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:48.899310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:48.921194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bfc0d-6db7-42ef-a994-eb2f1bbdf44a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:48.960450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:48.960974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:48.961255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:48.961528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:48.962188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:48.962497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:48.963874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:48.968737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 454/899] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:48.969510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.006476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.023382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa449d-99d1-49d3-8bd8-1b609888629b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.059618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.060221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.060505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.060778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.061413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:49.061718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.063117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.067855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 446/900] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:49.068638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.087302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf24f2-4cbb-4628-9b01-f557cf697e72 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.127203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.127809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.128132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.128407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.129641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:49.129978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.130691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.131847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.150430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 455/901] 10.222.0.61 () {56 vars in 1149 bytes} [Sat Sep 24 14:47:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:49.151637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.185371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e8818-7a9e-4bee-b55f-9dfbade9213f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.226551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.227172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.227456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.227722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.228439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] RBAC: Adding query filter params (interface=public, service_id=44f6d9b9d49a4ffdb57389fef6e15719, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:49.228745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.230160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.235747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 447/902] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/endpoints?service_id=44f6d9b9d49a4ffdb57389fef6e15719&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:49.236356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.254819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.265596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b835-69ec-47df-bf28-afef7bfb099d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.301882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.302605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.302882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.303156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.303772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:49.304069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.305432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.310572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 456/903] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:49.311328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.344757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0b42-0c30-4f90-978d-70ca93f80267 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.360555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.379125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.379732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.380017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.380300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.416283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30ffee-e9b9-463b-afd6-974502f10569 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.449821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.451295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.453983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.454589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.454869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.455139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.455733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.457106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.463362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 457/904] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:47:49.474952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.509026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.573782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 448/905] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:47:49.583690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.602790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.617946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756e00-cd5d-4951-8afc-d6a9444a53d0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.656044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.656988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.657467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.658002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.658927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] RBAC: Adding query filter params (interface=public, service_id=e318c6434bc54a24b212dea306518722, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:49.659453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.660872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.668666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 458/906] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/endpoints?service_id=e318c6434bc54a24b212dea306518722&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:49.674248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.712222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46ceb3-4725-44b4-b9e9-c6b2deb8a175 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.749064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.750082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.750566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.751024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.763921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.765678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:49.782940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:49.783908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:49.784791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:49.784791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:49.784791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:49.784791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 449/907] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:49.789801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:49.791916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.813467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ea144-b249-4b35-a008-e8d269c39c3c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.859761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.860476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.860903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.861840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:49.862033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.863988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.866153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.874956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 459/908] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:47:49.907940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d211a-e7b3-4ee3-890a-79b2b71d2f90 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:49.947228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:49.947855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:49.948137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:49.948414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:49.949081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:49.949387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:49.950830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:49.965039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 450/909] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:49.970748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:50.083629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836d086-30fa-48ee-8cc5-4f093ed4131d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:50.119883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:50.120500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:50.120781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:50.121049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:50.128180 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:50.129661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:50.141100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:50.142057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:50.143655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 460/910] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:50.149645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:50.261058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33798c0-b5c3-4b0c-9398-ff398828b46f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:50.290050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:50.298944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:50.299742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:50.300041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:50.300319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:50.301047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:50.301356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:50.302825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5bc0-8056-428c-b536-3199af80ecad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:50.315122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 451/911] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:50.322976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:50.335213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:50.437088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeeafd-3e5a-4b15-8014-b1ce72886b05 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:50.475398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:50.476685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:50.477113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:50.477550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:50.478467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] RBAC: Adding query filter params (role.id=997b25e6209744b2ae0f2086585686a4, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=82bd05cad09d4763a79fc7400921c3b0) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:50.478942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:50.480525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349160d-fe50-4701-83bb-b651fb609a90 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:50.486695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 461/912] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:47:50 2022] GET /identity/v3/role_assignments?user.id=82bd05cad09d4763a79fc7400921c3b0&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=997b25e6209744b2ae0f2086585686a4 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:50.536959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165b3d-40c0-442a-9065-e1f18df46f9e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:50.537249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165b3d-40c0-442a-9065-e1f18df46f9e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:50.537534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165b3d-40c0-442a-9065-e1f18df46f9e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:50.539292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 452/913] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:50.545166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:50.545455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:50.545736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:50.565728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:50.567654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:50.645677 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 462/914] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:50.922344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00165b3d-40c0-442a-9065-e1f18df46f9e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:51.032162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00165b3d-40c0-442a-9065-e1f18df46f9e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:51.065973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:51.066630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.066917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.067194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.067778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:51.069140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:51.077885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:51.078895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:51.079713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 453/915] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:51.087189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:51.089361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:51.196096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b8ca-119d-4250-979c-0149cfcde4ee None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:51.231309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:51.231971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.232303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.232579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.233207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:51.233508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:51.234975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:51.245367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 463/916] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:51.252177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:51.310651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:51.358982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad80980-beb6-4f7b-8ef1-4647c8a19e01 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:51.396858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:51.397467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.397749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.398052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.398688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:51.398995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:51.400346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:51.405545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 454/917] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:51.411596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:51.480203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:51.516555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19983acb-8fa7-4421-9014-952b2ac564f5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:51.555429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:51.556311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.556740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.557160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.558131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] RBAC: Adding query filter params (interface=public, service_id=0d6bcb8863a140cdb6bf84ac3abe6033, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:51.558590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:51.560127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd3eeb-c1ee-4486-8892-33dcda1c3a53 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:51.565824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 464/918] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:47:51 2022] GET /identity/v3/endpoints?service_id=0d6bcb8863a140cdb6bf84ac3abe6033&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:51.575208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:51.680600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9be76-c684-4cec-9143-ae59485380c6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:51.697111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:51.715215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c1e9-015b-4edd-aac4-16ac70537083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:51.715840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c1e9-015b-4edd-aac4-16ac70537083 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.716167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c1e9-015b-4edd-aac4-16ac70537083 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.716443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c1e9-015b-4edd-aac4-16ac70537083 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.717104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c1e9-015b-4edd-aac4-16ac70537083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:51.718560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c1e9-015b-4edd-aac4-16ac70537083 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:51.726368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 455/919] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:47:51 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:47:51.846257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53dbb-2447-4ab6-8dfc-4c8869e0b3d8 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.846437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53dbb-2447-4ab6-8dfc-4c8869e0b3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.846717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd53dbb-2447-4ab6-8dfc-4c8869e0b3d8 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.848363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 465/920] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:51.854282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2430a6-bce4-4600-b33c-4fc26c2ee57b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:51.854585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2430a6-bce4-4600-b33c-4fc26c2ee57b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:51.854845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2430a6-bce4-4600-b33c-4fc26c2ee57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:51.875772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aa2430a6-bce4-4600-b33c-4fc26c2ee57b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:51.876993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aa2430a6-bce4-4600-b33c-4fc26c2ee57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:51.959487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 456/921] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:52.193240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23b95b-e5b3-4650-9123-6e5caa3661e3 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.193581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23b95b-e5b3-4650-9123-6e5caa3661e3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.193704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23b95b-e5b3-4650-9123-6e5caa3661e3 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.195350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 466/922] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:52.200843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.201132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.201409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.224903 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:52.226188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:52.244557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23b95b-e5b3-4650-9123-6e5caa3661e3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:52.318036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 457/923] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:52.373142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23b95b-e5b3-4650-9123-6e5caa3661e3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:52.404585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:52.412012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:52.412915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.413407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.413878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.414713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:52.416270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:52.426295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:52.427201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:47:52.428065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:52.430789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 467/924] 10.222.0.61 () {56 vars in 1132 bytes} [Sat Sep 24 14:47:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:52.436785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:52.481509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:52.541513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660391b-3271-46ea-81e2-2613c1ead06e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:52.577978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91547078-4220-4f65-b514-269248f83f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:52.578960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91547078-4220-4f65-b514-269248f83f36 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.579443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91547078-4220-4f65-b514-269248f83f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.579925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91547078-4220-4f65-b514-269248f83f36 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.580754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91547078-4220-4f65-b514-269248f83f36 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:52.581255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91547078-4220-4f65-b514-269248f83f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:52.583017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91547078-4220-4f65-b514-269248f83f36 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:52.588314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 458/925] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:52.594151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:52.638596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91547078-4220-4f65-b514-269248f83f36 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:52.720779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83f456-950e-45fa-b249-8f7ac2063a91 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:52.753123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91547078-4220-4f65-b514-269248f83f36 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:52.757278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:52.757391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:52.757913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.758396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.758529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.759118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:52.759429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:52.760864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e20b52-eb67-42b9-9171-50427952e21f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:52.765800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 468/926] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:52.766964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21fc4a-861c-4994-8571-86b314562f28 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.767249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21fc4a-861c-4994-8571-86b314562f28 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.767526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21fc4a-861c-4994-8571-86b314562f28 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.770016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 469/927] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:52.774288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.774747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.775200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.793792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:52.795446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:52.796490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:52.797125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.797409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.797693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.798329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:52.800099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:52.809480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:52.814102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:52.814962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 459/928] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:52.815761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:52.859199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:52.874419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 470/929] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:52.875354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:52.940142 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b5931-4702-4f6a-a6a3-39f346b92a70 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:52.987347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:52.989515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:52.990016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:52.990767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:52.992275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:52.993444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f95636-97f4-44f2-829d-ed6589d34d30 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:52.994750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.030251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 460/930] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:47:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:53.051051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.051771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.052056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.052323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.052963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:53.053262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:53.054796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.060963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 471/931] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:53.071081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.104942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:53.155619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.193583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ceda5-d506-46bf-9b9a-c9674b59e8d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:53.214285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:53.233603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.234249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.234535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.234807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.235441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:53.235746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:53.237109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.243412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 461/932] 10.222.0.61 () {56 vars in 1114 bytes} [Sat Sep 24 14:47:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:53.248733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.282579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7793-b529-4055-b53d-8ba312eca1da None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:53.323672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.324431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.324780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.325128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.337056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:53.342030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:53.342947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:53.343765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 472/933] 10.222.0.61 () {56 vars in 1120 bytes} [Sat Sep 24 14:47:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:53.349528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.375956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:53.397657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa7ef1-52ff-4c99-892f-bc171d7b8617 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:53.446227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.447189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.447542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.447895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.449218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:53.451243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.467574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31f091-45d3-460c-a33d-810cf75bceb0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:53.479654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:53.483672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 462/934] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:47:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:47:53.523497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.524126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.524469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.524899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.525638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:53.525976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:53.527427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.538688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 473/935] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:47:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:53.544478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.662418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd501a3-adc7-4ca2-9ab7-2476ee7d1fce None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:53.699325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.700329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.700839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.701416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.713324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:53.715071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.717948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:53.731600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:53.732766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:53.733869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:53.733869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:53.733869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:53.738786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 463/936] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:53.742491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.849551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b92a33-8779-4150-ac41-7d5365aa0d71 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:53.887983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:53.888622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.888908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.889183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:53.889861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:53.890208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:53.891646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984b0e6-9ffd-4a76-96d3-7582214bf26d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:53.909460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 474/937] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:53.915420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:53.999122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c48bf-04e7-4ecc-a0f4-f86d1dee2d8b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:53.999472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c48bf-04e7-4ecc-a0f4-f86d1dee2d8b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:53.999709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c48bf-04e7-4ecc-a0f4-f86d1dee2d8b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.002477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 475/938] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:54.007118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.007404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.007679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.026885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:54.028230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:54.037923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655e9a1-a63b-47c9-b1c5-653b610eefa6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.053365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.075696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.076750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.077241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.077733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.086795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.088512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:54.101027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:54.102002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:54.102966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:54.109950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 464/940] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:47:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:54.110074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 476/940] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:47:54.110987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.212647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d6177-d772-4ec3-a511-aaa53039d3bf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.249277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.250223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.250730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.251285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.252228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:54.252747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.254441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.272629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 477/941] 10.222.0.61 () {56 vars in 1149 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:54.278633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.339489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.387325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f5812-29bf-4457-897e-57fd160bf9dc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.398681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.420374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.424691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.425331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.426269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.426549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.437972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.439433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:54.454748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:54.455713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:54.457015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:54.457015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:54.457015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:54.457015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 465/942] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:54.461654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.511411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad9209-56e3-4a16-b283-9e905df7f2f8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.547016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.547988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.548487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.548952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.549733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.551936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:54.561428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:54.562521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:54.563342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:54.571065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dd093-aa1f-483d-8fd1-aa7d601a1be0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.571479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 478/943] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:54.573012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.581937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.607891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.608507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.609004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.609342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.610033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:54.610380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.611867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.634921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 466/944] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:54.636053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.662328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.682588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9834f50-4e56-41a8-950a-38100d8abad6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.717503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.718417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.718838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.719249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.720007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:54.720450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.721984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.732190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 479/945] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:54.733539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.745844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913856ee-3135-4a3b-bd0b-6c6eda551f3f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.786176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.787094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.787374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.787648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.795135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.796626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:54.810196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:54.811077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:54.812113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 467/946] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:54.818404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.850318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.853351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc77434-0392-484e-8fab-e3b4b71d942d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.891428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.892572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.892999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.893448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.895308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:54.895624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.897006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:54.902681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 480/947] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:54.912221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:54.922146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:54.937206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32f66-15f1-4350-966b-8ab6c9c53ecc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:54.985401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:54.986044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:54.986376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:54.986588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:54.987283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:54.987869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:54.989311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:55.007530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 468/948] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:47:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:55.009343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:55.043190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9de9c-3a7f-4c94-8ba6-4c00f70c8389 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:55.079365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c9235-5345-4950-a096-dc3c9c4e583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:55.080317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c9235-5345-4950-a096-dc3c9c4e583f None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.080824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c9235-5345-4950-a096-dc3c9c4e583f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.081303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c9235-5345-4950-a096-dc3c9c4e583f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.082779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c9235-5345-4950-a096-dc3c9c4e583f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:55.084431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c9235-5345-4950-a096-dc3c9c4e583f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:55.104457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:55.110251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 481/949] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:47:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:47:55.126626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cfb55-b680-4a9b-8b9a-53b028f582e9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:55.173129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171a56b-7069-431a-9de4-5e05bf7e6a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:55.173711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171a56b-7069-431a-9de4-5e05bf7e6a45 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.174028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171a56b-7069-431a-9de4-5e05bf7e6a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.174302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171a56b-7069-431a-9de4-5e05bf7e6a45 None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/82bd05cad09d4763a79fc7400921c3b0/roles/997b25e6209744b2ae0f2086585686a4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.176641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171a56b-7069-431a-9de4-5e05bf7e6a45 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=82bd05cad09d4763a79fc7400921c3b0, role_id=997b25e6209744b2ae0f2086585686a4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:55.178144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171a56b-7069-431a-9de4-5e05bf7e6a45 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:55.194551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 469/950] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:47:55 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/82bd05cad09d4763a79fc7400921c3b0/roles/997b25e6209744b2ae0f2086585686a4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:55.348497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed2658-66ac-4528-8bf5-e0aa704f20fd None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.348885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed2658-66ac-4528-8bf5-e0aa704f20fd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.349082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed2658-66ac-4528-8bf5-e0aa704f20fd None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.350619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 482/951] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:55.356769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031fce-b5d0-401f-ae59-ec4d23e5c056 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.357114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031fce-b5d0-401f-ae59-ec4d23e5c056 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.357393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031fce-b5d0-401f-ae59-ec4d23e5c056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.380287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-48031fce-b5d0-401f-ae59-ec4d23e5c056 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:55.382883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-48031fce-b5d0-401f-ae59-ec4d23e5c056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:55.458924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 470/952] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:55.734427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed2658-66ac-4528-8bf5-e0aa704f20fd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:55.807975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:55.852804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed2658-66ac-4528-8bf5-e0aa704f20fd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:55.888926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:55.890426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.890918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.891418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.892216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:55.893782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-02db74ac-c149-4712-99aa-8c1f8e25b341 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:55.903195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:55.904219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:47:55.905064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:55.907300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 483/953] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:55.907631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f4485-b746-4223-8b1f-8a216ce93914 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.907945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f4485-b746-4223-8b1f-8a216ce93914 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.908278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f4485-b746-4223-8b1f-8a216ce93914 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.910240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 471/954] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:55.916000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f4485-b746-4223-8b1f-8a216ce93914 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:55.916147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:55.919561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:55.919561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:55.945970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:55.947342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:56.004091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:56.025986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 484/955] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:56.038841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f4485-b746-4223-8b1f-8a216ce93914 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:56.075311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:56.076236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:56.076755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:56.077237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:56.078131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:56.078611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:56.080193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:56.085891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 472/956] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:56.095186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:56.154928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:56.207774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62871c4-2e25-49fc-96c8-e793200be5b3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:56.244855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22488109-20d4-495a-82fa-76745605890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:56.245479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22488109-20d4-495a-82fa-76745605890d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:56.245760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22488109-20d4-495a-82fa-76745605890d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:56.246070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22488109-20d4-495a-82fa-76745605890d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:56.246701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22488109-20d4-495a-82fa-76745605890d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:56.247006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22488109-20d4-495a-82fa-76745605890d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:56.248370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22488109-20d4-495a-82fa-76745605890d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:56.253182 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 485/957] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:56.304673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:56.425083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afcf41-662c-464d-8c31-8e5bffcf4ca3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:56.460349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:56.460957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:56.461235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:56.461507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:56.462182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:56.463590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:56.466205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING keystone.server.flask.common [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 14:47:56.475076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:56.483536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 14:47:56.483536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 14:47:56.483536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [parameters: {'id': '19176cf19a4b49d2994f013854bc76c2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 14:47:56.483536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19176cf19a4b49d2994f013854bc76c2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19176cf19a4b49d2994f013854bc76c2. Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:47:56.491674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 14:47:56.492489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 24 14:47:56.493618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 14:47:56.495044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 14:47:56.495943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:47:56.496735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': '19176cf19a4b49d2994f013854bc76c2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:56.497689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:56.504085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19176cf19a4b49d2994f013854bc76c2. Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:56.506295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 473/958] 10.222.0.61 () {60 vars in 1140 bytes} [Sat Sep 24 14:47:56 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:56.509943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22488109-20d4-495a-82fa-76745605890d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:56.616002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22488109-20d4-495a-82fa-76745605890d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:56.637867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:56.653270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:56.654238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:56.654741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:56.655213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:56.662357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:56.664024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-51efeaf6-8491-454a-b353-b709f0e9a4d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:47:56.675670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:56.676659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:47:56.677621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:56.679923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 486/959] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:56.686143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:56.795254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397217-532a-44cc-85e1-d646429b9b36 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:56.830154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:56.830757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:56.831036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:56.831308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:56.832512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:56.832814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:56.834443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da8d5d-be6c-4346-bcd9-cd388d5e84f0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:56.855027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 474/960] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:47:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:57.394907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233cff-8269-4b7c-91e2-c3fc8f379715 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:57.395428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233cff-8269-4b7c-91e2-c3fc8f379715 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:57.395507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad233cff-8269-4b7c-91e2-c3fc8f379715 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:57.396914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 487/961] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:57.402578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca8930-4fdc-4e58-adce-bacb0259db10 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:57.403402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca8930-4fdc-4e58-adce-bacb0259db10 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:57.403880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca8930-4fdc-4e58-adce-bacb0259db10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:57.423486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e3ca8930-4fdc-4e58-adce-bacb0259db10 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:57.425416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3ca8930-4fdc-4e58-adce-bacb0259db10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:57.482505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:57.503584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 475/962] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:57.563649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8f1b-b22d-4469-a111-ac06ac78de99 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:57.563993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8f1b-b22d-4469-a111-ac06ac78de99 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:57.564206 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8f1b-b22d-4469-a111-ac06ac78de99 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:57.565779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 488/963] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:57.571537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:57.571862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:57.572106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:57.590766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:57.591952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:57.659392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 476/964] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:47:57.783482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8f1b-b22d-4469-a111-ac06ac78de99 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:57.892084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8f1b-b22d-4469-a111-ac06ac78de99 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:57.938081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:57.938251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:57.938761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:57.939046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:57.939321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:57.939908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:57.941339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:57.950928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:57.951971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:47:57.952826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 489/965] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:47:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:57.958304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.018306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.058152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7285d-8b87-4efb-9d6d-71337c1d8e5d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.074498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.089018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7017a-9119-42e0-a1d1-f93b4a72b050 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.096864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.098365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.098898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.099417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.110772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:47:58.116482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:58.117423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:47:58.118287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:58.125435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 477/966] 10.222.0.61 () {56 vars in 1120 bytes} [Sat Sep 24 14:47:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:58.126516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.130296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.130909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.131394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.131598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.132239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:58.132543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.134105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.139628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 490/967] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:58.145464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.241528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849897f-87fa-4cde-8f6f-6c3246409a42 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.258585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f97a2-aaf0-46e1-ae28-2b6b56042e43 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.268666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.281821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.282742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.283223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.283693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.284705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:58.285324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.287052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.300990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.300990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.300990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.300990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.301361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:58.301361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.301361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.301934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 478/968] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:58.303356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.305317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 491/969] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:58.311051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.341775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.415451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22aad8-fdee-4dbe-b1d2-cbf799f04799 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.427320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3618b49-971f-418b-9f67-ab4db34d05b3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.451768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.452484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.452864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.453307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.465451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.466386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.467086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.467395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.467986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.468459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.469451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] RBAC: Adding query filter params (interface=admin, service_id=0d6bcb8863a140cdb6bf84ac3abe6033, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:58.470131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.471651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.482119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 492/970] 10.222.0.61 () {56 vars in 1249 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/endpoints?service_id=0d6bcb8863a140cdb6bf84ac3abe6033&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:58.482421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:58.483300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:58.484237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:58.484237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:58.484237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:58.485562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.488547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 479/971] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:58.489572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.527037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.585136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.594552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c5d5-1e9c-4b57-a6f6-5611d5606092 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.598620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d301f6-cfbd-41b5-825e-d6895e042554 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.632774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429dee5-aa3d-43f2-b2ca-370eeaf31700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.633486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429dee5-aa3d-43f2-b2ca-370eeaf31700 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.633779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429dee5-aa3d-43f2-b2ca-370eeaf31700 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.634066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429dee5-aa3d-43f2-b2ca-370eeaf31700 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.634674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429dee5-aa3d-43f2-b2ca-370eeaf31700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.636047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429dee5-aa3d-43f2-b2ca-370eeaf31700 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.638637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.639382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.639664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.639936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.640569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:58.640870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.650213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 493/972] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:47:58.650419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.650535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bea0-d52a-4adf-ba16-9fd1de4fd9c4 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.650535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bea0-d52a-4adf-ba16-9fd1de4fd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.650535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bea0-d52a-4adf-ba16-9fd1de4fd9c4 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.650535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 494/973] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:58.651134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866df55-206e-44e9-96a2-53601192c100 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.651421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866df55-206e-44e9-96a2-53601192c100 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.651690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866df55-206e-44e9-96a2-53601192c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.666855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 480/974] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:58.673827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1866df55-206e-44e9-96a2-53601192c100 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:58.675014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.676444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1866df55-206e-44e9-96a2-53601192c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:58.771703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 495/975] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:47:58.808161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b31787-4db2-4e43-b9ff-ad25d680c586 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:58.846376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161d71-163c-48ba-a940-37251837318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:58.847008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161d71-163c-48ba-a940-37251837318f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:58.847355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161d71-163c-48ba-a940-37251837318f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:58.847632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161d71-163c-48ba-a940-37251837318f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:58.855197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a161d71-163c-48ba-a940-37251837318f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:58.856640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a161d71-163c-48ba-a940-37251837318f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6a161d71-163c-48ba-a940-37251837318f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:47:58.870408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:58.871417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:58.872360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 481/976] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:58.877303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866df55-206e-44e9-96a2-53601192c100 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:58.913071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:58.985769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866df55-206e-44e9-96a2-53601192c100 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.020749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.021360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.021633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.021915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.022662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:59.022967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.024628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:59.047420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161d71-163c-48ba-a940-37251837318f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.053207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 496/977] 10.222.0.61 () {56 vars in 1149 bytes} [Sat Sep 24 14:47:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:59.053774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.061469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.146298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.157220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322df89-fb8e-4785-b512-c1ab7f1fa380 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.158749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161d71-163c-48ba-a940-37251837318f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.192702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.193316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.193599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.193874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.199440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578a83-e8bf-4f35-9874-708653361ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.200046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578a83-e8bf-4f35-9874-708653361ea9 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.200325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578a83-e8bf-4f35-9874-708653361ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.200598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578a83-e8bf-4f35-9874-708653361ea9 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.201251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad578a83-e8bf-4f35-9874-708653361ea9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.203049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad578a83-e8bf-4f35-9874-708653361ea9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:59.225971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 482/978] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:47:59 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:47:59.225971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77939e74-93bc-4a1d-9c62-0f35d8e22d68 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.225971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77939e74-93bc-4a1d-9c62-0f35d8e22d68 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.225971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77939e74-93bc-4a1d-9c62-0f35d8e22d68 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.231775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 483/979] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:47:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:47:59.232656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.232954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.233233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.258188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:47:59.259485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:47:59.292734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.295271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:59.340993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.365933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 484/980] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:47:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:47:59.414052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.425230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 497/981] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:47:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:47:59.431502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.540345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa70113-5b52-4748-91ca-cee83f50fc6f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.576266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.577264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.577801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.578318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.596340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.597825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:47:59.614310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:47:59.615284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:47:59.616165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:47:59.616165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:47:59.616165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:59.616165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 485/982] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:47:59.622753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.646612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.702862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.741360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eee617-9b0a-44c1-aec1-5afdc0562811 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.765811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354be76a-a1f6-44db-81ac-a3ce958d59cd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.783547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.784183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.784642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.785020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.785665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:47:59.786286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.787674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:59.802921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 498/983] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:47:59.805794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.806451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.806738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.807228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.807673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.808526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.809103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:47:59.825462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 14:47:59.825462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 14:47:59.825462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [parameters: {'user_id': '3057f73c55e249f287dbfc110d7d7eb5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 14:47:59.825462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 24 14:47:59.835105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:47:59.836023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 14:47:59.836983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 14:47:59.837987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:47:59.838906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:47:59.839777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'user_id': '3057f73c55e249f287dbfc110d7d7eb5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:47:59.840635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 24 14:47:59.843181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 486/984] 10.222.0.61 () {60 vars in 1131 bytes} [Sat Sep 24 14:47:59 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:47:59.845698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:47:59.880807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.947758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8b28c-2abb-4a4f-9443-aae8ebbcdea3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.979923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594878ad-39b9-4b26-9961-71e0cb0e2bb6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:47:59.982694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:47:59.984047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:47:59.984047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:47:59.984047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:47:59.991312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:47:59.992105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:59.993141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:48:00.010142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:00.011105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:00.011993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 499/985] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:47:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:00.018847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:00.030237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:00.030993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:00.031279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:00.031556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:00.040751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:00.042697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:48:00.056603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:00.057675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:00.058959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 487/986] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:47:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:00.067625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:00.138151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3b25-e5ef-4481-8850-208a2fd26a9a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:00.148467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:00.176782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:00.177657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:00.178141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:00.178554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:00.179466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:00.179919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:00.181488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:00.194624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 500/987] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:48:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:00.197446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284310ef-a336-4ef3-9ec9-dcf84fda0a43 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:00.200451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:00.247599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:00.249049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:00.250088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:00.250826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:00.251741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:00.252491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:00.255460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44b90-0986-4df9-be1a-343001234035 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:00.308760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554e719-2223-49bf-8dcc-99bbf8a559f1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:00.312991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 488/988] 10.222.0.61 () {56 vars in 1101 bytes} [Sat Sep 24 14:48:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:00.349871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:00.350600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:00.350885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:00.351156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:00.351902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] RBAC: Adding query filter params (role.id=997b25e6209744b2ae0f2086585686a4, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=82bd05cad09d4763a79fc7400921c3b0) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:00.352210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:00.353755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89ec6a-3337-4e1e-829a-8bd48d880169 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:00.360986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 501/989] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:00 2022] GET /identity/v3/role_assignments?user.id=82bd05cad09d4763a79fc7400921c3b0&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=997b25e6209744b2ae0f2086585686a4 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:00.887454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c3a4-67a8-4f5e-9b5d-f56ab0d4f232 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:00.887976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c3a4-67a8-4f5e-9b5d-f56ab0d4f232 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:00.888087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c3a4-67a8-4f5e-9b5d-f56ab0d4f232 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:00.889618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 489/990] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:00.895428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:00.895825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:00.896016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:00.914347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:00.915518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:00.941654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:00.984057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 502/991] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:01.247869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c3a4-67a8-4f5e-9b5d-f56ab0d4f232 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:01.275557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:01.352808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c3a4-67a8-4f5e-9b5d-f56ab0d4f232 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:01.386550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:01.387170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:01.387446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:01.387716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:01.388290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:01.389622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:01.398184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:01.399192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:48:01.400136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:01.401989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 490/992] 10.222.0.61 () {56 vars in 1126 bytes} [Sat Sep 24 14:48:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:01.405411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:01.509424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997047c-1684-4d27-aec5-7505801c82fd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:01.543190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:01.543796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:01.544074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:01.544344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:01.544962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:01.545263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:01.546655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ae764-bddf-4158-9ecc-1c666df2550f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:01.551666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 503/993] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:48:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:01.557179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:01.585097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfb38-bb0a-4e02-9d7a-cae79ea6a29f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:01.585426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfb38-bb0a-4e02-9d7a-cae79ea6a29f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:01.585891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfb38-bb0a-4e02-9d7a-cae79ea6a29f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:01.587504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 504/994] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:01.592211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:01.592493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:01.592755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:01.611369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:01.612548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:01.643509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:01.662178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:01.674144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304de9ee-9a45-4323-980b-a4229890b29c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:01.709118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 505/995] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:01.715493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:01.716104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:01.716383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:01.716655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:01.717281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:01.717585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:01.718966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:01.724645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 491/996] 10.222.0.61 () {56 vars in 1122 bytes} [Sat Sep 24 14:48:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:01.730153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:01.830002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374956-383b-4a53-9550-ebea5c784e04 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:01.864842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:01.865457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:01.865738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:01.866042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:01.866708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:01.868068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:01.885538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 506/997] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:48:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:48:01.977713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:02.090507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6b561-f96e-4037-9f5c-f7c288313f26 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:02.131981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:02.132861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.133293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.133716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.134672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:02.136255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:02.146087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:02.147332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:48:02.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:02.150812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 492/998] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:48:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:02.156337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:02.176382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:02.263315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a2cb-8bfc-431b-975c-916cdcd33aaf None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:02.296945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:02.297549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.297826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.298123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.298749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:02.299054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:02.300962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:02.305962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 507/999] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:48:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:02.311340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:02.329190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:02.419037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b950c3b-8317-4900-b952-2d23cd37bbf6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:02.454069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:02.454709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.454996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.455342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.456001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:02.456337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:02.458368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eab3e6-078f-438d-835a-e6338b700904 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:02.469485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 493/1000] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:48:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:02.470468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:02.557699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:02.578353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cdac7-f5c1-44d8-bc4c-0237cfe29ae9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:02.579007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a78a9-2738-4c61-bea6-af77dedaa985 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.579297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a78a9-2738-4c61-bea6-af77dedaa985 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.579572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a78a9-2738-4c61-bea6-af77dedaa985 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.581047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 494/1001] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:02.586704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.586704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.586943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.606097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:02.607767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:02.621092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:02.621980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.622434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.622868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.623574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] RBAC: Adding query filter params (interface=public, service_id=55e6308b22994622b9a1afc09d343a47, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:02.623891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:02.625264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:02.630957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 508/1002] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:48:02 2022] GET /identity/v3/endpoints?service_id=55e6308b22994622b9a1afc09d343a47&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:02.637026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:02.679974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 495/1003] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:02.741182 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4014-c94d-4513-b70b-cbedc14e58f5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:02.775577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:02.776194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:02.776474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:02.776747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:02.777358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:02.778872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:02.786276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 509/1004] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:48:02 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:48:02.954333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:02.965969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:03.059930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76659afb-970a-4681-a09e-e1ca1015d9f7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:03.094052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:03.094686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:03.094968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:03.095244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:03.095835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:03.097245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:03.106134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:03.107127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:03.107978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:03.110022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 496/1005] 10.222.0.61 () {56 vars in 1105 bytes} [Sat Sep 24 14:48:02 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:03.114420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:03.125725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:03.217531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7d48e-3ee3-423e-92f0-a65ee7cb2830 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:03.251394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:03.252000 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:03.252281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:03.252555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:03.253255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] RBAC: Adding query filter params (name=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:03.253610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:03.255012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb3a06-6ba7-4690-84bf-ec3bc0bbb50a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:03.260152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 510/1006] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:03.265737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:03.335736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:03.367506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e46cf-e504-45e5-bf58-a08cf0e4d419 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:03.401254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:03.401858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:03.402171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:03.402454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:03.403085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:03.403397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:03.404751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd5ac3-041e-42b8-a8d5-c7509b025084 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:03.409305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 497/1007] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:04.072217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f37b-d2a0-43b0-821a-8033d01d9650 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:04.072566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f37b-d2a0-43b0-821a-8033d01d9650 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:04.072773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f37b-d2a0-43b0-821a-8033d01d9650 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:04.074194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 511/1008] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:04.079321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:04.079754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:04.079933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:04.098064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:04.099240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:04.166245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 498/1009] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:04.444215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f37b-d2a0-43b0-821a-8033d01d9650 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:04.515817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:04.547911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f37b-d2a0-43b0-821a-8033d01d9650 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:04.582334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae93497-18f7-4407-b840-15e1ea896f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:04.582956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae93497-18f7-4407-b840-15e1ea896f6d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:04.583234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae93497-18f7-4407-b840-15e1ea896f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:04.583501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae93497-18f7-4407-b840-15e1ea896f6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:04.584597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae93497-18f7-4407-b840-15e1ea896f6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:04.586232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae93497-18f7-4407-b840-15e1ea896f6d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:04.587956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 512/1010] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:04.593497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:04.662270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:04.698482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00553e81-59df-4656-ac4f-3ac30a523aba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:04.731462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04fee-74a1-4c4e-849c-8ed3050969bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:04.732068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04fee-74a1-4c4e-849c-8ed3050969bc None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:04.732351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04fee-74a1-4c4e-849c-8ed3050969bc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:04.732627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04fee-74a1-4c4e-849c-8ed3050969bc None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:04.733284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd04fee-74a1-4c4e-849c-8ed3050969bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:04.734737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd04fee-74a1-4c4e-849c-8ed3050969bc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:04.769937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 499/1011] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:48:04 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:48:04.958348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f277ce0-b6f5-4984-8748-b0fa385021eb None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:04.958637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f277ce0-b6f5-4984-8748-b0fa385021eb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:04.958828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f277ce0-b6f5-4984-8748-b0fa385021eb None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:04.960164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 513/1012] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:04.965499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f159e96-4876-4179-8322-943b14827bae None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:04.965797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f159e96-4876-4179-8322-943b14827bae None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:04.966118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f159e96-4876-4179-8322-943b14827bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:04.985252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0f159e96-4876-4179-8322-943b14827bae None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:04.986501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f159e96-4876-4179-8322-943b14827bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:05.056778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 500/1013] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:05.323649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f277ce0-b6f5-4984-8748-b0fa385021eb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:05.426884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f277ce0-b6f5-4984-8748-b0fa385021eb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:05.462234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:05.462920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:05.463202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:05.463477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:05.464134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:05.465562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-23c57056-a4ee-469d-8824-357db26c095a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:05.476292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:05.477346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:05.478251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 514/1014] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:48:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:05.485361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f159e96-4876-4179-8322-943b14827bae None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:05.535376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:05.578065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718f6c6-ea0b-4696-998b-4d895658caac None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:05.578353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718f6c6-ea0b-4696-998b-4d895658caac None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:05.578559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718f6c6-ea0b-4696-998b-4d895658caac None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:05.579893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 515/1015] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:05.585487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:05.585774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:05.586078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:05.589135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f159e96-4876-4179-8322-943b14827bae None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:05.593170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:05.606828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:05.608415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:05.627286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:05.628228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:05.629063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:05.629375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:05.630043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:05.630351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:05.632196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:05.637124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 501/1016] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:48:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:05.642504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:05.679396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 516/1017] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:05.745325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378fd17-0fbc-44e8-8715-0c78091992c1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:05.758870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:05.781144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:05.781750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:05.782065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:05.782342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:05.783071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:05.783410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:05.784797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:05.789744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 502/1018] 10.222.0.61 () {56 vars in 1112 bytes} [Sat Sep 24 14:48:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:05.795396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:05.893015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74440ac0-69e4-4aa4-8646-144bb61a67c9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:05.935298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe823e-fbf2-44c4-9810-36d593e7f0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:05.935911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe823e-fbf2-44c4-9810-36d593e7f0a3 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:05.936191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe823e-fbf2-44c4-9810-36d593e7f0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:05.936461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe823e-fbf2-44c4-9810-36d593e7f0a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:05.937134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe823e-fbf2-44c4-9810-36d593e7f0a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:05.938558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe823e-fbf2-44c4-9810-36d593e7f0a3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:05.943579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:05.954106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:05.961348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 517/1019] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:48:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:06.069180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf7d65-a3bf-47e5-a2f2-f22313c04c5e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:06.115035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd772bc-bbb9-4343-b83e-6d4f946005f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:06.115973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd772bc-bbb9-4343-b83e-6d4f946005f7 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:06.116475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd772bc-bbb9-4343-b83e-6d4f946005f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:06.116936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd772bc-bbb9-4343-b83e-6d4f946005f7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:06.117791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd772bc-bbb9-4343-b83e-6d4f946005f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:06.119353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd772bc-bbb9-4343-b83e-6d4f946005f7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:06.136967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 503/1020] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:48:05 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:48:06.962760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfdc66-dc1b-494d-ba8f-62dac7c1e0f0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:06.963286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfdc66-dc1b-494d-ba8f-62dac7c1e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:06.963286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfdc66-dc1b-494d-ba8f-62dac7c1e0f0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:06.964651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 518/1021] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:06.970798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:06.971693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:06.972177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:06.990845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:06.992269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:07.019451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:07.076283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 504/1022] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:07.342052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfdc66-dc1b-494d-ba8f-62dac7c1e0f0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:07.449338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfdc66-dc1b-494d-ba8f-62dac7c1e0f0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:07.483290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03216a9c-d3f8-4629-9d5a-4b817a92dbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:07.483911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03216a9c-d3f8-4629-9d5a-4b817a92dbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:07.484185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03216a9c-d3f8-4629-9d5a-4b817a92dbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:07.484453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03216a9c-d3f8-4629-9d5a-4b817a92dbd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:07.485546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03216a9c-d3f8-4629-9d5a-4b817a92dbd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:07.486998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03216a9c-d3f8-4629-9d5a-4b817a92dbd7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:07.489521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 519/1023] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:07.495000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:07.591039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9bd6-00f5-40d7-b502-c2c31d4c979a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:07.624283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dc004-9fb8-4fee-a7e0-e94513ff0659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:07.624822 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dc004-9fb8-4fee-a7e0-e94513ff0659 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:07.625099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dc004-9fb8-4fee-a7e0-e94513ff0659 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:07.625366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dc004-9fb8-4fee-a7e0-e94513ff0659 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:07.626030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dc004-9fb8-4fee-a7e0-e94513ff0659 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:07.627436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dc004-9fb8-4fee-a7e0-e94513ff0659 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:07.649317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 505/1024] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:48:07 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:08.303695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b4124-0ab6-4eee-b681-c34a3b5c0d3d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:08.304258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b4124-0ab6-4eee-b681-c34a3b5c0d3d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:08.304258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b4124-0ab6-4eee-b681-c34a3b5c0d3d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:08.305533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 520/1025] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:08.310981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:08.311402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:08.311547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:08.330589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:08.331845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:08.341501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:08.372812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6932-d851-416c-8128-fc7c85b0bca3 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:08.373089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6932-d851-416c-8128-fc7c85b0bca3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:08.373596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a6932-d851-416c-8128-fc7c85b0bca3 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:08.374973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 521/1026] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:08.381202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:08.381713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:08.382292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:08.402253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:08.403455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:08.418234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 506/1027] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:08.458776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:08.475736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 522/1028] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:08.682721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:08.749755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:08.792419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af0422-401f-4ed9-abba-db628e3f5717 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:08.829138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:08.829767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:08.830080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:08.830359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:08.841328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:08.846226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:08.847752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:08.848631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 507/1029] 10.222.0.61 () {56 vars in 1120 bytes} [Sat Sep 24 14:48:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:08.852892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f28e04-6291-4223-b599-b7ded6dc5ead None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:08.854431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:08.865943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:08.896168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:08.896827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:08.897107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:08.897379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] PATH_INFO: `/v3/services/image` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:08.897990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:08.899360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:08.909145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:08.910992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:08.911940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 523/1030] 10.222.0.61 () {56 vars in 1105 bytes} [Sat Sep 24 14:48:08 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:08.917455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:08.975855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:08.979451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0b744-e6a9-40ec-8fbd-80dcb200e6ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.017619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.018251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.018530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.019320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.019987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:09.020291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.021640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.029952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 508/1031] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:48:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:09.031821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41827d32-5a75-4a9b-94a8-c2c04a43c895 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.036253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.069521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.070358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.070660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.071009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.071634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:09.072054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.073428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.078579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 524/1032] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:09.084568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.102205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:09.139043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517719-2750-4e73-92a0-5662b04d094f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.173700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.174367 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.174646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.174912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.186758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.188188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.188364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f560cb-9be8-416f-a477-c692b2e864e6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:09.202907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:09.203869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:09.204774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:09.204774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:09.204774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:09.205998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:09.209165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 509/1033] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:09.210444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.224650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.225144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.225476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.225752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.226475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:09.226757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.228109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.232990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 525/1034] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:09.239415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.324394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d168c1a-429c-4548-b7b3-c4263379bf59 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.341497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f8b09-50e3-4782-8b03-0c6df310b6f5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.343247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:09.361852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.362490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.363066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.363349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.363988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:09.364319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.365679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.377602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 510/1035] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:09.379097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.379705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.379977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.380242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.380934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] RBAC: Adding query filter params (interface=public, service_id=d0571bd4b8cf4cab8d2b9f6188e95128, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:09.381242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.382628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.383469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.392519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 526/1036] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/endpoints?service_id=d0571bd4b8cf4cab8d2b9f6188e95128&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:09.393468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.442808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:09.490877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54078d1-87c7-4308-9ccc-9dfbf48e477e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.500854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4bb37-f224-4765-bd6c-5e727a90acec None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.529092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.530060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.530346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.530761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.537285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.537845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.537992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.538185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.538453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.539050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.539291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.540405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.547334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 527/1037] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:48:09.551104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:09.552070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:09.558336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 511/1038] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:09.558970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.668066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd3659-e273-4bb1-b0fd-a92c805ab726 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.678361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:09.706077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.707254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.707530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.707800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.708530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:09.708832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.710296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1fb0e8-c498-4436-a453-2da06bbd7c7f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.729826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 528/1039] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:09.735827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:09.786314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:09.847999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef46a8e-34c5-4565-b4d1-83d83afda929 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:09.864371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b055-60cd-4c31-9b05-f1fa8ea1f7d2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.864970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b055-60cd-4c31-9b05-f1fa8ea1f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.865260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b055-60cd-4c31-9b05-f1fa8ea1f7d2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.866768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 529/1040] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:09.872862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.873389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.873765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.883999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:09.884614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:09.884898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:09.885271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:09.893463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:09.895150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:09.961230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:09.962717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:09.975780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 530/1041] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:10.007266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.074754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 512/1042] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:10.080421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.175715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.183150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a1ba2-1a1a-45ca-b1c8-46fa6a9bc5da None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.216481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.217119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.217396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.217668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.228298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.229750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:10.245230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:10.251272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:10.252296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:10.252296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:10.252296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:10.252296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 531/1043] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:10.252524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.255456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.359496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a33dc-63bd-470e-9028-93cbf8c68868 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.364102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724309b-4e77-4ba1-81dd-750c8cf8dbdc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.398908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.399326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.399953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.400117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.400254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.400529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.400826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.401266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.402084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:10.402542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.404060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.410981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.411945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:10.416909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 513/1044] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:10.417881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:10.418841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:10.419697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:10.422548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.423868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 532/1045] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:10.424784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.439021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.530378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4c856-fc7a-4257-94f1-d64a82f3c26c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.533276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9e95c-5d25-4a4a-9625-bb00019a89af None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.565254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.566519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.566804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.567073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.567725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:10.568026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.568775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.569387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.569493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.569667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.569961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.576977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.578339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 533/1046] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:10.578518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.583891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:48:10.590235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:10.591168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:10.592087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 514/1047] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:10.596796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.658333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.659843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.703813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624d2c-5c73-41c8-8b67-b4b3da81bdd2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.704137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aca34a-6eed-483f-a935-da698e3bb348 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.740203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.740803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.740932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.741085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.741359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.741625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.742007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.742115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:10.742374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] PATH_INFO: `/v3/users/d7e85011a7ad4a929df1e998edfe595e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.742468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.743822 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.758251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 515/1048] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:10.758762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e85011a7ad4a929df1e998edfe595e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.760453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.762612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 534/1049] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/users/d7e85011a7ad4a929df1e998edfe595e => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:10.764263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.768945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:10.868733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dd582-5390-4cf0-ae0d-9d024e0f7031 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.874581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dca83-d55a-46a1-9267-7c91637eea28 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:10.887394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.901005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:10.907753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.908410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.908647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.908926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.909196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:10.910251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] RBAC: Adding query filter params (role.id=997b25e6209744b2ae0f2086585686a4, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=dc85d332eeaf4c649f51858fa608101f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:10.910371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:10.910562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.910667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:10.910865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:10.911996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:10.933937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 516/1050] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/role_assignments?user.id=dc85d332eeaf4c649f51858fa608101f&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=997b25e6209744b2ae0f2086585686a4 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:10.995611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:10.997071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:11.102987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 535/1051] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:48:11.109772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:11.212272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305fdb-288c-4182-84c6-6b0bca2f3a1f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:11.246262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca987-d0ba-4dd6-bbfa-47c7115f1eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:11.246920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca987-d0ba-4dd6-bbfa-47c7115f1eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:11.247207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca987-d0ba-4dd6-bbfa-47c7115f1eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:11.247486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca987-d0ba-4dd6-bbfa-47c7115f1eb9 None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:11.248598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ca987-d0ba-4dd6-bbfa-47c7115f1eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=73674a33d098421399d12641f6da6657)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:11.250061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ca987-d0ba-4dd6-bbfa-47c7115f1eb9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:11.251874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 517/1052] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:11 2022] GET /identity/v3/projects/73674a33d098421399d12641f6da6657 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:11.258007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:11.286694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:11.360384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74002d78-ff30-4161-ae89-8412b763c4a9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:11.396651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:11.397279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:11.397526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:11.397799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:11.398559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=73674a33d098421399d12641f6da6657, user.id=d7e85011a7ad4a929df1e998edfe595e) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:11.398871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:11.400305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe483dfa-f87c-4d97-a06f-0ff00366c612 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:11.406049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 536/1053] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:11 2022] GET /identity/v3/role_assignments?user.id=d7e85011a7ad4a929df1e998edfe595e&scope.project.id=73674a33d098421399d12641f6da6657&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:11.715496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9fd8e-96b3-40d4-b2ce-c31df5ccae68 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:11.716042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9fd8e-96b3-40d4-b2ce-c31df5ccae68 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:11.716337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9fd8e-96b3-40d4-b2ce-c31df5ccae68 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:11.717877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 518/1054] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:11.723424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:11.723715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:11.723985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:11.742514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:11.743699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:11.813954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 537/1055] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:12.079079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9fd8e-96b3-40d4-b2ce-c31df5ccae68 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:12.135541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:12.183504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9fd8e-96b3-40d4-b2ce-c31df5ccae68 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:12.216450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:12.217079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:12.217354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:12.217631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:12.218248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:12.219613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:12.228262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:12.229242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:12.230616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 519/1056] 10.222.0.61 () {56 vars in 1105 bytes} [Sat Sep 24 14:48:12 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:12.235305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:12.260445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:12.343325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1c7f3-a81c-4423-b268-9918784925d4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:12.384020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:12.384630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:12.384909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:12.385182 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:12.385807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] RBAC: Adding query filter params (name=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:12.386142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:12.387507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:12.397023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 538/1057] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:12.399013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:12.505413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad00e0c-1d39-4847-aafb-d726e306ec6e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:12.519481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:12.540980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:12.541581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:12.541860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:12.542156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:12.542786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:12.543089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:12.544445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee9d0b-fef3-47a3-bd72-741c7df416b2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:12.554571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 520/1058] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:12.556169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:12.641865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:12.657886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891872-65f5-4162-b858-6119b6375e98 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:12.693773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693970b-24a2-49fa-af63-84398a79eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:12.694415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693970b-24a2-49fa-af63-84398a79eba9 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:12.694698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693970b-24a2-49fa-af63-84398a79eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:12.694968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693970b-24a2-49fa-af63-84398a79eba9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:12.695645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693970b-24a2-49fa-af63-84398a79eba9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:12.697013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693970b-24a2-49fa-af63-84398a79eba9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:12.716369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 539/1059] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:48:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:48:13.116558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767d03d-72f1-491a-9744-ab0480bb7bad None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.117090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767d03d-72f1-491a-9744-ab0480bb7bad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.117090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767d03d-72f1-491a-9744-ab0480bb7bad None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.118679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 521/1060] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:13.123675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a2ed-c371-40f9-ae48-f4f82a8a1b36 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.124026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a2ed-c371-40f9-ae48-f4f82a8a1b36 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.124367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176a2ed-c371-40f9-ae48-f4f82a8a1b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.142235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1176a2ed-c371-40f9-ae48-f4f82a8a1b36 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:13.143406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1176a2ed-c371-40f9-ae48-f4f82a8a1b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:13.216004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 540/1061] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:13.481486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767d03d-72f1-491a-9744-ab0480bb7bad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:13.542844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2976bb-a9b5-42e9-9cf6-921076b3335f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.543364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2976bb-a9b5-42e9-9cf6-921076b3335f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.543364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2976bb-a9b5-42e9-9cf6-921076b3335f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.544693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 541/1062] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:13.550552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.551048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.551495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.569631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:13.571419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:13.586239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767d03d-72f1-491a-9744-ab0480bb7bad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:13.626448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:13.627217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.627503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.627781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.639030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:13.642065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:13.646255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 542/1063] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:13.652211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:13.653157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:13.654619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 522/1064] 10.222.0.61 () {56 vars in 1120 bytes} [Sat Sep 24 14:48:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:13.659377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:13.760136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f9e83-1b64-46d8-b7e2-22edc3b38b34 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:13.794910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:13.795517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.795802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.796076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.796735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:13.797101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:13.798505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:13.806698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 543/1065] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:48:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:13.812327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:13.853273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:13.912931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:13.916136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36231c3a-f838-4eda-8caf-3d520ef639fd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:13.958981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:13.959988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:13.960269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:13.960866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:13.972091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:13.973539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:13.978341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:13.989008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:13.989995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:13.995653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:13.995653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:13.995653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:13.995653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 523/1066] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:13.997281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.026414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cc46-ce00-4c05-aab9-b8d99ca816ba None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.061374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.062021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.062301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.062572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.074205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:14.078974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:14.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:14.080892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 544/1067] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:14.086572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.098892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.110251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee07f10-830e-4648-a420-e6f7e6ed3f61 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.149403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.150051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.150355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.150663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.151316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:14.151676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.153043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.167133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 524/1068] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:14.173131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.190878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302f996-472a-4399-a2d7-0a813faaf937 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.210525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.227386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.228409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.228698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.228974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.229786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:14.230141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.231509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.241628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 545/1069] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:14.247056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.278329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ce9e7-7d28-4e5e-b8cf-a92e0967f385 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.316644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.317283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.317570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.317846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.325297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.326777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:48:14.339591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:14.340550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:14.341483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 525/1070] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:14.346543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.354843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa75b8-b0b8-4d5e-a9d0-71abca9b939d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.360028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.390810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.391337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.391621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.391892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] PATH_INFO: `/v3/users/d7e85011a7ad4a929df1e998edfe595e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.393235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e85011a7ad4a929df1e998edfe595e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.395268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.401447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 546/1071] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/users/d7e85011a7ad4a929df1e998edfe595e => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:14.402433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.452962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc448e9-e62e-4769-82c1-81b9bf3382c7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.454984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.490012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.490627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.490909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.491183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.491912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:14.492330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.493643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.508230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833de09b-561f-4eab-8c80-2ecdb53cf942 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.516728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 526/1072] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:14.517785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.545964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.546498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.546785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.547058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.548178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] RBAC: Authorizing `identity:get_project(project_id=73674a33d098421399d12641f6da6657)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.549597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.551493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 547/1073] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/projects/73674a33d098421399d12641f6da6657 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:14.557874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.591061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.627658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae5192-8072-442a-bf06-6b68095e3571 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.662096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3cef6-d469-4701-8a10-f305588ca000 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.667645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.668532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.669036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.669927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.679446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.682991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.685832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.698522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.699198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.699479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.699754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657/users/d7e85011a7ad4a929df1e998edfe595e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.701686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] RBAC: Authorizing `identity:create_grant(project_id=73674a33d098421399d12641f6da6657, user_id=d7e85011a7ad4a929df1e998edfe595e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.703153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:14.705533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:14.707199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:14.708148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:14.708148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:14.708148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:14.709702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 527/1074] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:14.715769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.720823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 548/1075] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:48:14 2022] PUT /identity/v3/projects/73674a33d098421399d12641f6da6657/users/d7e85011a7ad4a929df1e998edfe595e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:48:14.833507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374989b-7dd4-4e6b-9c08-09f60fff06db None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:14.855268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:14.872341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:14.872966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.873330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.873521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.874245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:14.874555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:14.875961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ac6dc-8776-493a-b026-df23e938dbf0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:14.894614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 528/1076] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:14.897925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:14.914918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413c6c-b625-4938-af67-d65f521a6b44 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.915209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413c6c-b625-4938-af67-d65f521a6b44 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.915493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413c6c-b625-4938-af67-d65f521a6b44 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.916991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 529/1077] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:14.927319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:14.927514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:14.927792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:14.954849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:14.957079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:15.017939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ce54-bf1b-4797-b49e-23568f2c3e1a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:15.051912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:15.053965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:15.053965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:15.053965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:15.058033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 530/1078] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:15.069039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:15.070547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:48:15.084470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:15.085376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:15.086655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 549/1079] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:15.091628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:15.218417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f158be2-8c72-40af-ae4a-87463794cdbc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:15.241011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:15.260036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:15.260961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:15.261485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:15.262018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:15.262946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:15.263457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:15.265146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:15.277678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 531/1080] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:15.285879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:15.328774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:15.342953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:15.393643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17438846-7230-47e9-bac5-42a86b65025e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:15.427275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:15.427934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:15.428214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:15.428489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/dc85d332eeaf4c649f51858fa608101f/roles/997b25e6209744b2ae0f2086585686a4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:15.430675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=dc85d332eeaf4c649f51858fa608101f, role_id=997b25e6209744b2ae0f2086585686a4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:15.432141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:15.445972 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 550/1081] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:48:15 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/dc85d332eeaf4c649f51858fa608101f/roles/997b25e6209744b2ae0f2086585686a4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:48:15.456391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef2a8e-7a66-40e5-a876-b1361de1629f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:15.496278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:15.497190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:15.497632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:15.498093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:15.542841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:15.574791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:15.576699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:15.686226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 532/1082] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:48:15.692258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:15.798802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:15.804950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe72b-02c9-49da-8cfc-82d5a01a0edc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:15.840015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:15.840923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:15.841388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:15.841847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:15.853153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:15.854820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8d805728-abb5-467a-9fc2-556055333fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:15.869341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:15.870341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:15.871259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:15.871259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:15.871259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:15.871259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 551/1083] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:15.877321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:15.953949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:15.987648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad661d-fed3-4bc7-b4a0-5ea66fdf1c17 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:16.022623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:16.023230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:16.023511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:16.023783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:16.024419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:16.024786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:16.026119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4261e-33d7-498c-b785-fbac69ab3e18 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:16.038773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 533/1084] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:16.916007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c3cde-f749-4bad-88e0-8e6d72a03a97 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:16.916656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c3cde-f749-4bad-88e0-8e6d72a03a97 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:16.916656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c3cde-f749-4bad-88e0-8e6d72a03a97 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:16.918121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 552/1085] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:16.923340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:16.923755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:16.923898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:16.943167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:16.944355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:17.008420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 534/1086] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:17.274104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c3cde-f749-4bad-88e0-8e6d72a03a97 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:17.374059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c3cde-f749-4bad-88e0-8e6d72a03a97 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:17.406607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:17.407235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:17.407512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:17.407786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:17.418660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:17.423413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:17.425990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:17.427171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 553/1087] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:48:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:17.430548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:17.487159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:17.531062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075383-3782-48d0-907e-e25e345fe662 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:17.564352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:17.564957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:17.565239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:17.565571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:17.566280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:17.566589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:17.567975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe65d5a-21d6-41f7-9e60-1d06fbf957cd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:17.575806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 535/1088] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:48:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:17.582214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:17.594730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:17.613573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3f89a-d3ca-4c06-ba39-739862bdd2e8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:17.613862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3f89a-d3ca-4c06-ba39-739862bdd2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:17.614168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3f89a-d3ca-4c06-ba39-739862bdd2e8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:17.615636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 536/1089] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:17.620291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:17.620574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:17.620851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:17.638860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:17.640041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:17.686184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee0ba4-4c04-47bb-87b1-138e0df78958 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:17.708528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 537/1090] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:17.719052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:17.719669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:17.719943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:17.720208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] PATH_INFO: `/v3/users/d7e85011a7ad4a929df1e998edfe595e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:17.721297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] RBAC: Authorizing `identity:get_user(user_id=d7e85011a7ad4a929df1e998edfe595e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:17.722751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:17.724408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 554/1091] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:17 2022] GET /identity/v3/users/d7e85011a7ad4a929df1e998edfe595e => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:17.729971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:17.780998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:17.826413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66a2e-08c3-467c-9c5f-c3c11f091852 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:17.859008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:17.859609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:17.859894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:17.860168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:17.924509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:17.926317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:17.975565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:17.978022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.024216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 538/1092] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:18.030003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.054934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:18.077470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d518703-3334-4413-a428-cbe4a7c2846f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.110287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.110905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.111186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.111458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.122278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:18.127148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76a00a-41fe-4f37-b37d-fcc06c8bc212 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:18.127505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:18.129052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:18.130416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 555/1093] 10.222.0.61 () {56 vars in 1120 bytes} [Sat Sep 24 14:48:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:18.135021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.163357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.163980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.164257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.164533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.165628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] RBAC: Authorizing `identity:get_project(project_id=73674a33d098421399d12641f6da6657)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.167078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.168990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 539/1094] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/projects/73674a33d098421399d12641f6da6657 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:18.174675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.196718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:18.232121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9200e-5020-4034-8833-71087692a9b6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.265517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.266134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.266412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.266680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.267328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:18.267628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.269507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.273879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4800fb-cd21-4b77-99b4-8be56dcd6099 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.274287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:18.281932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 556/1095] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:18.284053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.308751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.309352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.309635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.309930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.310664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=73674a33d098421399d12641f6da6657, user.id=d7e85011a7ad4a929df1e998edfe595e) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:18.311052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.312542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.320026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 540/1096] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/role_assignments?user.id=d7e85011a7ad4a929df1e998edfe595e&scope.project.id=73674a33d098421399d12641f6da6657&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:18.396107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37540fb8-85db-4fca-bfa5-72c39588769d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.431333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.431990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.432272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.432546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.444510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.446027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:18.462495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:18.463625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:18.464547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:18.464547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:18.464547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:18.464547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 557/1097] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:48:18.469719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.556607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:18.573171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20424963-e87e-45b6-a8fd-516ced74b064 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.605651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.606278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.606556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.606826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.607460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:18.607761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.609106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.620937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 541/1098] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:18.626594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.678491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:18.725311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc705-43cf-4766-8858-c81210920dc4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.758475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975df147-a342-4031-9499-79a06506561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.759095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df147-a342-4031-9499-79a06506561e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.759377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df147-a342-4031-9499-79a06506561e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.759649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df147-a342-4031-9499-79a06506561e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.769324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975df147-a342-4031-9499-79a06506561e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.770784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975df147-a342-4031-9499-79a06506561e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-975df147-a342-4031-9499-79a06506561e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:48:18.781700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:18.782872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:18.784239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 558/1099] 10.222.0.61 () {56 vars in 1093 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:18.788629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:18.886165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c994e0-94bc-4a25-95c7-bd1752d452b7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:18.920146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:18.920767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:18.921062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:18.921342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:18.922110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:18.922422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:18.923853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:18.941171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 542/1100] 10.222.0.61 () {56 vars in 1133 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:18.946885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975df147-a342-4031-9499-79a06506561e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:19.043834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975df147-a342-4031-9499-79a06506561e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:19.056234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:19.076838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:19.077440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:19.077713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:19.078015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:19.143231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:19.145293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:19.240518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 559/1101] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:19.246336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:19.249203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:19.343834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717381f-8792-432c-abea-8c3e9fbcdc98 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:19.376434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:19.377050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:19.377328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:19.377598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:19.387928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:19.389358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:19.403989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:19.405031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:48:19.405944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:19.405944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:48:19.405944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:19.405944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 543/1102] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:19.410819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:19.424459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:19.509544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaddc8-3b1a-4a2c-9551-bd0cbfa05191 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:19.543146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:19.543741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:19.544020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:19.544292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:19.544925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:19.545228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:19.546624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:19.558591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 560/1103] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:19.564051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:19.620165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:19.665342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1abbe-4b02-43ce-b8c9-8864a0f50364 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:19.699258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:19.699895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:19.700198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:19.700474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:19.707272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:19.708723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:48:19.720094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:19.721153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:19.722105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 544/1104] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:19.727101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:19.795699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:19.826539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaf28f-8b81-4422-b784-bb8a759df73f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:19.859109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:19.859717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:19.859996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:19.860266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:19.860965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:19.861292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:19.863615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405ffe8-998d-4038-987e-2fd096998c94 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:19.875271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 561/1105] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:19.881974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:19.980749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79444902-400d-414a-8d2d-47ffe7b70768 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:19.994942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:20.014898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:20.015491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:20.015767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:20.016035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:20.016754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] RBAC: Adding query filter params (role.id=997b25e6209744b2ae0f2086585686a4, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=dc85d332eeaf4c649f51858fa608101f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:20.017058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:20.018506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676f8c7-a9e3-45f0-8625-71b339eae740 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:20.024038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 545/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:19 2022] GET /identity/v3/role_assignments?user.id=dc85d332eeaf4c649f51858fa608101f&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=997b25e6209744b2ae0f2086585686a4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:20.535609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf2a65-4ff7-46b7-989a-0814cb2fd7dc None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:20.536150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf2a65-4ff7-46b7-989a-0814cb2fd7dc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:20.536150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf2a65-4ff7-46b7-989a-0814cb2fd7dc None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:20.537488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 562/1107] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:20.542906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:20.543199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:20.543476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:20.561678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:20.562917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:20.632383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 546/1108] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:20.900445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf2a65-4ff7-46b7-989a-0814cb2fd7dc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:20.999569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf2a65-4ff7-46b7-989a-0814cb2fd7dc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:21.028100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:21.033325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60871f06-9f03-4b78-8c1e-b969c36abded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:21.033967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60871f06-9f03-4b78-8c1e-b969c36abded None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:21.034246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60871f06-9f03-4b78-8c1e-b969c36abded None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:21.034513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60871f06-9f03-4b78-8c1e-b969c36abded None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:21.035614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60871f06-9f03-4b78-8c1e-b969c36abded None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:21.037033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60871f06-9f03-4b78-8c1e-b969c36abded None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:21.038596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 563/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:21.044308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:21.139973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f7de-5cd1-4794-a23d-f7fc722101de None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:21.148801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:21.172602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521ebc7-6a19-401c-b8e2-9355c035aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:21.173137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521ebc7-6a19-401c-b8e2-9355c035aa5b None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:21.173417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521ebc7-6a19-401c-b8e2-9355c035aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:21.173688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521ebc7-6a19-401c-b8e2-9355c035aa5b None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:21.174366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521ebc7-6a19-401c-b8e2-9355c035aa5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:21.176298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521ebc7-6a19-401c-b8e2-9355c035aa5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:21.196328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 547/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:48:21 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:23.374243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57202-d61e-4c80-8131-a81ff2bd3caf None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:23.374788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57202-d61e-4c80-8131-a81ff2bd3caf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:23.374788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57202-d61e-4c80-8131-a81ff2bd3caf None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:23.376094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 564/1111] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:23.381310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:23.381702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:23.381860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:23.400107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:23.401282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:23.465045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 548/1112] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:23.740345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57202-d61e-4c80-8131-a81ff2bd3caf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:23.844321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57202-d61e-4c80-8131-a81ff2bd3caf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:23.876089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:23.876702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:23.876976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:23.877241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] PATH_INFO: `/v3/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:23.879121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] RBAC: Authorizing `identity:get_role(role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:23.880468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:23.881757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 565/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:23 2022] GET /identity/v3/roles/c374f9c914654fa8aad1636ed6f12185 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:23.887582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:23.976227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:23.983547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62163320-9ed1-4e65-a442-f085c819ddb2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:24.017665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:24.018250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:24.018534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:24.018808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] PATH_INFO: `/v3/users/ab03a0677dcd403693a9d3063629a7ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:24.031399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] RBAC: Authorizing `identity:get_user(user_id=ab03a0677dcd403693a9d3063629a7ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:24.032827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:24.034437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 549/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:23 2022] GET /identity/v3/users/ab03a0677dcd403693a9d3063629a7ed => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:24.040186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:24.108641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:24.142331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb705d8-0b94-49c6-97c8-30372d3e9b23 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:24.173966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:24.174555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:24.174829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:24.175097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:24.240220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:24.241662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:24.302725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:24.338221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 566/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:24.344168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:24.438774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a2896-17e2-4f79-9412-4ae1968409fa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:24.470416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd819-f4fa-4f3f-8b62-c777eab89869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:24.471021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd819-f4fa-4f3f-8b62-c777eab89869 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:24.471298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd819-f4fa-4f3f-8b62-c777eab89869 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:24.471565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd819-f4fa-4f3f-8b62-c777eab89869 None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:24.473206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcd819-f4fa-4f3f-8b62-c777eab89869 None admin] RBAC: Authorizing `identity:get_project(project_id=73674a33d098421399d12641f6da6657)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:24.474636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcd819-f4fa-4f3f-8b62-c777eab89869 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:24.476221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 550/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:24 2022] GET /identity/v3/projects/73674a33d098421399d12641f6da6657 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:24.482159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:24.576496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb00a7-48fe-4805-a663-1132ed708e85 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:24.608944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:24.609533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:24.609850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:24.610118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:24.610824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] RBAC: Adding query filter params (role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=73674a33d098421399d12641f6da6657, user.id=ab03a0677dcd403693a9d3063629a7ed) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:24.611129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:24.612539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6160008-ea88-47e4-86b6-20d0fbea2174 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:24.618565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 567/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:24 2022] GET /identity/v3/role_assignments?user.id=ab03a0677dcd403693a9d3063629a7ed&scope.project.id=73674a33d098421399d12641f6da6657&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:26.702118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644116e0-588e-48aa-8da5-60a422aa0c42 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:26.702647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644116e0-588e-48aa-8da5-60a422aa0c42 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:26.702647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644116e0-588e-48aa-8da5-60a422aa0c42 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:26.704007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 551/1118] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:26.709564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:26.710001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:26.710132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:26.728498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:26.729689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:26.755698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:26.795472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 568/1119] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:27.064483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644116e0-588e-48aa-8da5-60a422aa0c42 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:27.086183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:27.162102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644116e0-588e-48aa-8da5-60a422aa0c42 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:27.195364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:27.195980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:27.196261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:27.196536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] PATH_INFO: `/v3/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:27.197932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] RBAC: Authorizing `identity:get_role(role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:27.199287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:27.200596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 552/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:27 2022] GET /identity/v3/roles/c374f9c914654fa8aad1636ed6f12185 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:27.206339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:27.301927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5c311-75d8-490c-aa5f-416eb8c1b917 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:27.335005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:27.335563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:27.335841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:27.336112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] PATH_INFO: `/v3/users/ab03a0677dcd403693a9d3063629a7ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:27.337219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=ab03a0677dcd403693a9d3063629a7ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:27.338674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:27.340226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 569/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:27 2022] GET /identity/v3/users/ab03a0677dcd403693a9d3063629a7ed => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:27.346143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:27.430325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:27.441887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194eb06-433b-4df9-be88-87ae05bf62f5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:27.474119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf777d9-844f-4907-abab-9f726c952bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:27.474730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf777d9-844f-4907-abab-9f726c952bad None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:27.475009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf777d9-844f-4907-abab-9f726c952bad None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:27.475282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf777d9-844f-4907-abab-9f726c952bad None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:27.476385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf777d9-844f-4907-abab-9f726c952bad None admin] RBAC: Authorizing `identity:get_project(project_id=73674a33d098421399d12641f6da6657)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:27.477794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf777d9-844f-4907-abab-9f726c952bad None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:27.479412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 553/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:27 2022] GET /identity/v3/projects/73674a33d098421399d12641f6da6657 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:27.485267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:27.514599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:27.579918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df7b7c-be8a-4933-8d60-2190e4fcc3e7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:27.612441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7d598-dee3-4b98-a285-66824d9b67de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:27.613564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7d598-dee3-4b98-a285-66824d9b67de None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:27.613836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7d598-dee3-4b98-a285-66824d9b67de None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:27.614128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7d598-dee3-4b98-a285-66824d9b67de None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657/users/ab03a0677dcd403693a9d3063629a7ed/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:27.616208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7d598-dee3-4b98-a285-66824d9b67de None admin] RBAC: Authorizing `identity:create_grant(project_id=73674a33d098421399d12641f6da6657, user_id=ab03a0677dcd403693a9d3063629a7ed, role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:27.617618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7d598-dee3-4b98-a285-66824d9b67de None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:27.629876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 570/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:48:27 2022] PUT /identity/v3/projects/73674a33d098421399d12641f6da6657/users/ab03a0677dcd403693a9d3063629a7ed/roles/c374f9c914654fa8aad1636ed6f12185 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:48:29.733338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d7b45-34ad-483c-98ed-eccf40340325 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:29.733865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d7b45-34ad-483c-98ed-eccf40340325 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:29.733985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d7b45-34ad-483c-98ed-eccf40340325 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:29.735254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 554/1124] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:29.741072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:29.741507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:29.741624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:29.759679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:29.760863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:29.830868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 571/1125] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:30.100888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d7b45-34ad-483c-98ed-eccf40340325 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:30.197024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d7b45-34ad-483c-98ed-eccf40340325 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:30.229761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:30.230401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:30.230683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:30.230955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] PATH_INFO: `/v3/roles/c374f9c914654fa8aad1636ed6f12185` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:30.232319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] RBAC: Authorizing `identity:get_role(role_id=c374f9c914654fa8aad1636ed6f12185)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:30.233670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:30.234980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 555/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:30 2022] GET /identity/v3/roles/c374f9c914654fa8aad1636ed6f12185 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:30.240853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:30.260797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:30.336350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1473fe9-a6b8-4269-a907-b48189c5fc59 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:30.369928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:30.370478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:30.370755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:30.371023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] PATH_INFO: `/v3/users/ab03a0677dcd403693a9d3063629a7ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:30.372117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] RBAC: Authorizing `identity:get_user(user_id=ab03a0677dcd403693a9d3063629a7ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:30.373542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:30.375120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 572/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:30 2022] GET /identity/v3/users/ab03a0677dcd403693a9d3063629a7ed => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:30.381421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:30.406209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:30.476755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872c653-5b9a-4a87-a250-c22876a89049 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:30.509178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:30.509762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:30.510075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:30.510347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:30.574161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:30.575584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:30.598374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:30.669477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 556/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:30.675290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:30.760354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:30.772503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58786280-2df1-4e09-b606-3933814c2368 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:30.804912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5096056-bb91-4860-b2f7-78a21990381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:30.805521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5096056-bb91-4860-b2f7-78a21990381f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:30.805800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5096056-bb91-4860-b2f7-78a21990381f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:30.806105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5096056-bb91-4860-b2f7-78a21990381f None admin] PATH_INFO: `/v3/projects/73674a33d098421399d12641f6da6657` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:30.807204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5096056-bb91-4860-b2f7-78a21990381f None admin] RBAC: Authorizing `identity:get_project(project_id=73674a33d098421399d12641f6da6657)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:30.808626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5096056-bb91-4860-b2f7-78a21990381f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:30.810244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 573/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:30 2022] GET /identity/v3/projects/73674a33d098421399d12641f6da6657 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:30.817070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:30.912038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfe0bf-1b71-4ba4-82d3-bd829ebdef24 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:30.934875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:30.945964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:30.946558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:30.946833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:30.947103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:30.947819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] RBAC: Adding query filter params (role.id=c374f9c914654fa8aad1636ed6f12185, scope.project.id=73674a33d098421399d12641f6da6657, user.id=ab03a0677dcd403693a9d3063629a7ed) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:30.948124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:30.949533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608836d-f25a-44f0-9158-5fb4c3d244a7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:30.955082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 557/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:30 2022] GET /identity/v3/role_assignments?user.id=ab03a0677dcd403693a9d3063629a7ed&scope.project.id=73674a33d098421399d12641f6da6657&role.id=c374f9c914654fa8aad1636ed6f12185 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:48:33.189431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f80e2-2930-4acf-a7d3-ceace0c3dd86 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:33.189960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f80e2-2930-4acf-a7d3-ceace0c3dd86 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:33.189960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f80e2-2930-4acf-a7d3-ceace0c3dd86 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:33.191289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 574/1131] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:33.196505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:33.196883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:33.197074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:33.215160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:33.216322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:33.279077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 558/1132] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:33.542190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f80e2-2930-4acf-a7d3-ceace0c3dd86 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:33.638970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f80e2-2930-4acf-a7d3-ceace0c3dd86 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:33.671659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0b372-4ab1-4b07-88b1-11f7261f5afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:33.672268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0b372-4ab1-4b07-88b1-11f7261f5afd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:33.672549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0b372-4ab1-4b07-88b1-11f7261f5afd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:33.672821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0b372-4ab1-4b07-88b1-11f7261f5afd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:33.673940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0b372-4ab1-4b07-88b1-11f7261f5afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:33.675372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0b372-4ab1-4b07-88b1-11f7261f5afd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:33.676897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 575/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:33.682755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:33.780666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69045fa4-02aa-4c75-ad9d-7cbbc119a4e3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:33.799874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:33.817144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccb84c-e093-49ca-bba8-89e8af00bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:33.817745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccb84c-e093-49ca-bba8-89e8af00bd84 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:33.818050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccb84c-e093-49ca-bba8-89e8af00bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:33.818322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccb84c-e093-49ca-bba8-89e8af00bd84 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:33.818973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccb84c-e093-49ca-bba8-89e8af00bd84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:33.820391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccb84c-e093-49ca-bba8-89e8af00bd84 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:33.845203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 559/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:48:33 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:35.983733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763ee9-46e2-4a1c-91f1-67aaabe2077e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:35.984177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763ee9-46e2-4a1c-91f1-67aaabe2077e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:35.984385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763ee9-46e2-4a1c-91f1-67aaabe2077e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:35.985775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 576/1135] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:35.991947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:35.992361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:35.992508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:36.010520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:36.011705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:36.085114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 560/1136] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:36.355362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763ee9-46e2-4a1c-91f1-67aaabe2077e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:36.380581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:36.454821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763ee9-46e2-4a1c-91f1-67aaabe2077e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:36.488213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f5016-6458-41a3-86d1-d4a7adffea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:36.488837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5016-6458-41a3-86d1-d4a7adffea73 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:36.489118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5016-6458-41a3-86d1-d4a7adffea73 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:36.489389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5016-6458-41a3-86d1-d4a7adffea73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:36.490504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f5016-6458-41a3-86d1-d4a7adffea73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:36.491933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f5016-6458-41a3-86d1-d4a7adffea73 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:36.493474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 577/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:48:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:36.499264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:36.591885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:36.595200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47a3e-0ce5-4ff0-bfc7-2047dd1042b9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:36.631012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da112a6-451d-4fdc-b280-82ab33e3c333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:36.631540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da112a6-451d-4fdc-b280-82ab33e3c333 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:36.631820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da112a6-451d-4fdc-b280-82ab33e3c333 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:36.632094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da112a6-451d-4fdc-b280-82ab33e3c333 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:36.632755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da112a6-451d-4fdc-b280-82ab33e3c333 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:36.634235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da112a6-451d-4fdc-b280-82ab33e3c333 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:36.654615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 561/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:48:36 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:38.815214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2bf68-d49c-4a62-8457-5a94279d0cbc None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:38.816331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2bf68-d49c-4a62-8457-5a94279d0cbc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:38.816331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2bf68-d49c-4a62-8457-5a94279d0cbc None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:38.817611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 578/1139] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:48:38.823729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:38.824172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:38.824291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:38.842927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:38.844119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:38.909195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 562/1140] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:39.181691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2bf68-d49c-4a62-8457-5a94279d0cbc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:39.270280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:39.279502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2bf68-d49c-4a62-8457-5a94279d0cbc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:39.312022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:39.312637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:39.312916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:39.313196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:39.323557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:39.328302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:39.329392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:39.330454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 579/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:39.335366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:39.415932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:39.431675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3cbb1-e62c-4dca-863d-6bab15c8b3db None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:39.465487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:39.466038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:39.466319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:39.466590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:39.467248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:39.467550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:39.468903 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:39.477188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 563/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:39.483090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:39.579434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2d1f2-a9f6-4c3e-a324-be9f3b861a1e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:39.612271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:39.613862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:39.614205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:39.614477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] PATH_INFO: `/v3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:39.627081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] RBAC: Authorizing `identity:get_user(user_id=09afc4cddcda4ccfab4d9ffedd4d8f6f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:39.628562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:39.630359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 580/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:39 2022] GET /identity/v3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:39.635902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:39.731970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd21f44-845e-4862-8c53-5108adab6e5b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:39.764756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:39.765345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:39.765630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:39.765926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:39.783068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:39.831857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:39.833298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:39.927653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 564/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:39.933394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:39.945308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:40.028337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a1a3e-d22a-41d7-9927-db4ed748189e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:40.060431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f81f8-e129-4cc0-9122-8200185cc1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:40.061038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f81f8-e129-4cc0-9122-8200185cc1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:40.061310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f81f8-e129-4cc0-9122-8200185cc1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:40.061574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f81f8-e129-4cc0-9122-8200185cc1e0 None admin] PATH_INFO: `/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:40.062674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f81f8-e129-4cc0-9122-8200185cc1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=3adea28942d44dc5baabf3fac1cfbbb3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:40.064122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f81f8-e129-4cc0-9122-8200185cc1e0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:40.065710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 581/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:39 2022] GET /identity/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:40.071911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:40.119194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:40.168332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec5651-65ad-45e8-b1fb-00f5c411043b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:40.200868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:40.201454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:40.201740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:40.202046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:40.202766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=3adea28942d44dc5baabf3fac1cfbbb3, user.id=09afc4cddcda4ccfab4d9ffedd4d8f6f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:40.203079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:40.204499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a358eb-d3d9-4e60-bb49-bc528bcabb0a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:40.210259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 565/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:40 2022] GET /identity/v3/role_assignments?user.id=09afc4cddcda4ccfab4d9ffedd4d8f6f&scope.project.id=3adea28942d44dc5baabf3fac1cfbbb3&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:42.300029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64b4e3-42bc-4e15-85e9-b94da9a37a16 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:42.300569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64b4e3-42bc-4e15-85e9-b94da9a37a16 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:42.300569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64b4e3-42bc-4e15-85e9-b94da9a37a16 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:42.301929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 582/1147] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:42.306922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:42.307351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:42.307454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:42.326123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:42.327315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:42.393578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 566/1148] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:42.659934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64b4e3-42bc-4e15-85e9-b94da9a37a16 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:42.736287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:42.759424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64b4e3-42bc-4e15-85e9-b94da9a37a16 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:42.791251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:42.791865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:42.792147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:42.792419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:42.802795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:42.807545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:42.808682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:42.809513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 583/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:48:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:42.814856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:42.855874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:42.910003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379f714-05d2-434c-8aa6-215d4519c012 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:42.943102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:42.943702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:42.943980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:42.944251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:42.944908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:42.945212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:42.946591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:42.954295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 567/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:48:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:42.960031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:43.056526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad95a-a95d-430c-b1cb-82ac97a9b99e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:43.089590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:43.090230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:43.090516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:43.090792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] PATH_INFO: `/v3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:43.091946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] RBAC: Authorizing `identity:get_user(user_id=09afc4cddcda4ccfab4d9ffedd4d8f6f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:43.093379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:43.094971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 584/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:42 2022] GET /identity/v3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:43.101551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:43.197947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af4d46-6438-4eb9-8107-5966b3f531fd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:43.200095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:43.231035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb80b62-5049-47ec-b4c0-f25107aad5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:43.231657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80b62-5049-47ec-b4c0-f25107aad5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:43.231942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80b62-5049-47ec-b4c0-f25107aad5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:43.232217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80b62-5049-47ec-b4c0-f25107aad5e7 None admin] PATH_INFO: `/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:43.233325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80b62-5049-47ec-b4c0-f25107aad5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=3adea28942d44dc5baabf3fac1cfbbb3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:43.234776 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80b62-5049-47ec-b4c0-f25107aad5e7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:43.236403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 568/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:43 2022] GET /identity/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:43.242457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:43.251012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:43.344215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34963d33-6b85-419e-9128-6c67f1cd2a93 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:43.376573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37593388-2687-4ed6-b44f-6ae992ab1085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:43.377219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37593388-2687-4ed6-b44f-6ae992ab1085 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:43.377503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37593388-2687-4ed6-b44f-6ae992ab1085 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:43.377778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37593388-2687-4ed6-b44f-6ae992ab1085 None admin] PATH_INFO: `/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:43.380275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37593388-2687-4ed6-b44f-6ae992ab1085 None admin] RBAC: Authorizing `identity:create_grant(project_id=3adea28942d44dc5baabf3fac1cfbbb3, user_id=09afc4cddcda4ccfab4d9ffedd4d8f6f, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:43.381712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37593388-2687-4ed6-b44f-6ae992ab1085 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:43.395378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 585/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:48:43 2022] PUT /identity/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:48:45.505454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d73bf-748c-40c3-b52c-eedd4aad6468 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:45.506052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d73bf-748c-40c3-b52c-eedd4aad6468 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:45.506052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d73bf-748c-40c3-b52c-eedd4aad6468 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:45.507375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 569/1154] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:45.512594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:45.512961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:45.513142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:45.531969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:45.533130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:45.588912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:45.596811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 586/1155] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:45.869507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d73bf-748c-40c3-b52c-eedd4aad6468 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:45.968981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d73bf-748c-40c3-b52c-eedd4aad6468 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:46.001281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:46.001928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:46.002215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:46.002491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:46.013313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:48:46.018140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:46.019234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:48:46.020287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 570/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:48:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:46.025233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:46.118917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f9527-a33a-4aeb-a23d-7a652a58e894 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:46.152057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:46.152589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:46.152870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:46.153143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:46.153798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:46.154141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:46.155505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:46.163744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 587/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:48:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:46.169384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:46.200292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:46.266801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15032ab7-9f7e-469a-acef-d00450778037 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:46.299140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:46.299753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:46.300582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:46.300858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] PATH_INFO: `/v3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:46.301990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] RBAC: Authorizing `identity:get_user(user_id=09afc4cddcda4ccfab4d9ffedd4d8f6f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:46.303412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:46.304966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 571/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:48:46 2022] GET /identity/v3/users/09afc4cddcda4ccfab4d9ffedd4d8f6f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:46.310621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:46.367117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:46.408465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e935c2-2569-4d63-a85b-b2a119bd107b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:46.441321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:46.441942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:46.442228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:46.442505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:46.510530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:46.511983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:46.564858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:46.605291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 588/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:46.611583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:46.706558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:46.709325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd999b6-59c8-45b6-b362-2976d9bdee00 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:46.744631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9294a-0741-45c0-8390-8f3adb0c2102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:46.745250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9294a-0741-45c0-8390-8f3adb0c2102 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:46.745530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9294a-0741-45c0-8390-8f3adb0c2102 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:46.745802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9294a-0741-45c0-8390-8f3adb0c2102 None admin] PATH_INFO: `/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:46.746935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9294a-0741-45c0-8390-8f3adb0c2102 None admin] RBAC: Authorizing `identity:get_project(project_id=3adea28942d44dc5baabf3fac1cfbbb3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:46.748343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9294a-0741-45c0-8390-8f3adb0c2102 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:46.749983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 572/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:48:46 2022] GET /identity/v3/projects/3adea28942d44dc5baabf3fac1cfbbb3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:46.756820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:46.850978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896db2-4b06-4ce5-b987-f2ade84bf9c2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:46.884428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:46.885016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:46.885296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:46.885567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:46.886306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=3adea28942d44dc5baabf3fac1cfbbb3, user.id=09afc4cddcda4ccfab4d9ffedd4d8f6f) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:46.886613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:46.888034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f17a04-e406-4b2a-a587-18bb212a1192 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:46.893544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 589/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:48:46 2022] GET /identity/v3/role_assignments?user.id=09afc4cddcda4ccfab4d9ffedd4d8f6f&scope.project.id=3adea28942d44dc5baabf3fac1cfbbb3&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:49.040341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47790a3d-63ce-473c-afc7-0960c7f0d1d3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:49.040914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47790a3d-63ce-473c-afc7-0960c7f0d1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:49.040914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47790a3d-63ce-473c-afc7-0960c7f0d1d3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:49.042260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 573/1162] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:49.047571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:49.047960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:49.048138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:49.065625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:49.066819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:49.082823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:49.138267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 590/1163] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:48:49.400628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47790a3d-63ce-473c-afc7-0960c7f0d1d3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:49.498039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47790a3d-63ce-473c-afc7-0960c7f0d1d3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:49.529627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:49.530237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:49.530515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:49.530784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:49.562938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:49.594706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:49.596123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:49.690068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 574/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:48:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:48:49.696544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:49.790488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59c168-1fd6-40fe-bff2-65a5c2fd170b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:49.823609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:49.824222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:49.824500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:49.824771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:49.831273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:49.832714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-41d0a1d1-a7dd-407d-b40e-29ae6ed9a47a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:48:49.843782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:48:49.845026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:48:49.846158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 591/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:48:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:48:49.850742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:49.914678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:49.947568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251dc4c-0682-417a-b965-bd07e10dd1cc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:49.979956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:49.980550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:49.980831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:49.981117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:49.981736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:48:49.982068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:49.983416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc7d9c-3a10-4e02-ab5b-499255850913 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:49.991220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 575/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Sat Sep 24 14:48:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:52.117126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26660-a11a-4754-9144-289335854489 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:52.117661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26660-a11a-4754-9144-289335854489 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:52.117661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26660-a11a-4754-9144-289335854489 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:52.119038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 592/1167] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:52.124286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd53b3-9eac-4cb3-9d51-248b4c9709ee None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:52.124734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd53b3-9eac-4cb3-9d51-248b4c9709ee None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:52.124836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd53b3-9eac-4cb3-9d51-248b4c9709ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:52.143276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c1dd53b3-9eac-4cb3-9d51-248b4c9709ee None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:52.145120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c1dd53b3-9eac-4cb3-9d51-248b4c9709ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:52.209195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 576/1168] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:52.475188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc26660-a11a-4754-9144-289335854489 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:52.490393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:52.573669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc26660-a11a-4754-9144-289335854489 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:52.605373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab791b1c-60fd-46ee-9a27-0d76bb09bc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:52.605982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab791b1c-60fd-46ee-9a27-0d76bb09bc91 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:52.606262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab791b1c-60fd-46ee-9a27-0d76bb09bc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:52.606534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab791b1c-60fd-46ee-9a27-0d76bb09bc91 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:52.607186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab791b1c-60fd-46ee-9a27-0d76bb09bc91 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:52.608558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab791b1c-60fd-46ee-9a27-0d76bb09bc91 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:52.624562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 593/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Sat Sep 24 14:48:52 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:54.774750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b373bc6-55fa-40ab-8618-a4b73aa817e3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:54.775299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b373bc6-55fa-40ab-8618-a4b73aa817e3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:54.775574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b373bc6-55fa-40ab-8618-a4b73aa817e3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:54.776951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 577/1170] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:54.782165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:54.782595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:54.782724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:54.801208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:54.802418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:54.866091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 594/1171] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:55.133354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b373bc6-55fa-40ab-8618-a4b73aa817e3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:55.193629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:55.231360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b373bc6-55fa-40ab-8618-a4b73aa817e3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:55.263712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea246e-89cc-4046-9e8f-c24be4c13418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:55.264317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea246e-89cc-4046-9e8f-c24be4c13418 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:55.264596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea246e-89cc-4046-9e8f-c24be4c13418 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:55.264868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea246e-89cc-4046-9e8f-c24be4c13418 None admin] PATH_INFO: `/v3/domains/f3897d76c3f74662b6d027d21b446707` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:55.275444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea246e-89cc-4046-9e8f-c24be4c13418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f3897d76c3f74662b6d027d21b446707)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:55.276872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea246e-89cc-4046-9e8f-c24be4c13418 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:55.278578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 578/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Sat Sep 24 14:48:55 2022] GET /identity/v3/domains/f3897d76c3f74662b6d027d21b446707 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:55.284255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:55.286516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:55.380457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd309a73-17aa-4d29-89ce-c6826310cebd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:55.412602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2af78-7bbe-47f5-bfb5-1dd61c0aa25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:55.413130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2af78-7bbe-47f5-bfb5-1dd61c0aa25f None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:55.413404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2af78-7bbe-47f5-bfb5-1dd61c0aa25f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:55.413672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2af78-7bbe-47f5-bfb5-1dd61c0aa25f None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:55.414333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2af78-7bbe-47f5-bfb5-1dd61c0aa25f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:55.415747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2af78-7bbe-47f5-bfb5-1dd61c0aa25f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:55.440199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 595/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:48:55 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:48:57.586784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754a47-c1b9-4dcc-a0bf-1fb56ce2737c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:57.587308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754a47-c1b9-4dcc-a0bf-1fb56ce2737c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:57.587308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754a47-c1b9-4dcc-a0bf-1fb56ce2737c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:57.588662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 579/1174] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:48:57.594078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:57.594494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:57.594637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:57.612401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:48:57.613848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:48:57.661468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:57.679153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 596/1175] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:48:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:48:57.958541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754a47-c1b9-4dcc-a0bf-1fb56ce2737c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:58.057258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754a47-c1b9-4dcc-a0bf-1fb56ce2737c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:58.078745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:48:58.089815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a61a-1ed6-49cd-892a-da73261893d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:58.090457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a61a-1ed6-49cd-892a-da73261893d1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:58.090737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a61a-1ed6-49cd-892a-da73261893d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:58.091011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a61a-1ed6-49cd-892a-da73261893d1 None admin] PATH_INFO: `/v3/domains/f3897d76c3f74662b6d027d21b446707` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:58.092112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103a61a-1ed6-49cd-892a-da73261893d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f3897d76c3f74662b6d027d21b446707)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:58.093534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103a61a-1ed6-49cd-892a-da73261893d1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:58.095124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 580/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Sat Sep 24 14:48:57 2022] GET /identity/v3/domains/f3897d76c3f74662b6d027d21b446707 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:48:58.100887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:48:58.197377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d6b5c-8a6b-421d-9fe6-c919e1860164 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:48:58.229543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae68f74-ba2a-40c2-9045-3aa19e7d5648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:48:58.230151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae68f74-ba2a-40c2-9045-3aa19e7d5648 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:48:58.230433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae68f74-ba2a-40c2-9045-3aa19e7d5648 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:48:58.230702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae68f74-ba2a-40c2-9045-3aa19e7d5648 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:48:58.231344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae68f74-ba2a-40c2-9045-3aa19e7d5648 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:48:58.232767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae68f74-ba2a-40c2-9045-3aa19e7d5648 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:48:58.253592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 597/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:48:58 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:49:00.407244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de206224-11bf-4afc-a17f-a81e5dbe6bdc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:00.407790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de206224-11bf-4afc-a17f-a81e5dbe6bdc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:00.407790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de206224-11bf-4afc-a17f-a81e5dbe6bdc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:00.409654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 581/1178] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:49:00.415120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:00.415552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:00.415655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:00.438017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:49:00.439206 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:49:00.472380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:00.505282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 598/1179] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:49:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:49:00.776529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de206224-11bf-4afc-a17f-a81e5dbe6bdc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:00.873611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de206224-11bf-4afc-a17f-a81e5dbe6bdc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:00.905823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:00.906458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:00.906735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:00.907008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:00.917097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:49:00.921817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:49:00.922950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:49:00.923997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 582/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:49:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:49:00.929378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:01.025216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824eb8a8-4103-4a83-90d0-028c81d72d21 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:01.058652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:01.059507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:01.059943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:01.060388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:01.061507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:49:01.062067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:01.064393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:01.072769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 599/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:49:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:01.078804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:01.127472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:01.175760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7f539-f936-4965-a561-41e7d785e03d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:01.206791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:01.207396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:01.207681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:01.207950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] PATH_INFO: `/v3/users/3500edb403994da08f5cda3b3f55ab44` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:01.220293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] RBAC: Authorizing `identity:get_user(user_id=3500edb403994da08f5cda3b3f55ab44)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:01.222261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:01.223817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 583/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:49:01 2022] GET /identity/v3/users/3500edb403994da08f5cda3b3f55ab44 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:01.229601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:01.260341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:01.325686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfcad4-e6da-48e5-a011-90985786832f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:01.357405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:01.358017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:01.358300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:01.358573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:01.422279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:01.423710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:01.452510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:01.519507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 600/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:49:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:49:01.525304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:01.619643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e2add-f649-49a2-a2ef-6d8190e2c356 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:01.628352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:01.652230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7551-903a-4cb8-b014-4b27d88a2acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:01.652833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7551-903a-4cb8-b014-4b27d88a2acc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:01.653108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7551-903a-4cb8-b014-4b27d88a2acc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:01.653377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7551-903a-4cb8-b014-4b27d88a2acc None admin] PATH_INFO: `/v3/projects/eeccd2f4121e415dba609ae4d2d831ab` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:01.654487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c7551-903a-4cb8-b014-4b27d88a2acc None admin] RBAC: Authorizing `identity:get_project(project_id=eeccd2f4121e415dba609ae4d2d831ab)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:01.655906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c7551-903a-4cb8-b014-4b27d88a2acc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:01.657489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 584/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:49:01 2022] GET /identity/v3/projects/eeccd2f4121e415dba609ae4d2d831ab => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:01.664109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:01.759779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6390cf-3c6e-4b0e-93d5-52ba78824d18 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:01.790174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:01.795592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:01.796192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:01.796471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:01.796750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:01.797466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=eeccd2f4121e415dba609ae4d2d831ab, user.id=3500edb403994da08f5cda3b3f55ab44) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:49:01.797775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:01.799238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dd948-a5f7-4fe5-ba53-7e02c37966b7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:01.804402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 601/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:49:01 2022] GET /identity/v3/role_assignments?user.id=3500edb403994da08f5cda3b3f55ab44&scope.project.id=eeccd2f4121e415dba609ae4d2d831ab&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:03.889474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a880-2cef-4312-b4dd-f1df2786c1af None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:03.890025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a880-2cef-4312-b4dd-f1df2786c1af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:03.890119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a880-2cef-4312-b4dd-f1df2786c1af None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:03.891394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 585/1186] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:49:03.897559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:03.897997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:03.898133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:03.916493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:49:03.917683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:49:03.982349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 602/1187] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:49:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:49:04.249825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42a880-2cef-4312-b4dd-f1df2786c1af None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:04.347963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42a880-2cef-4312-b4dd-f1df2786c1af None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:04.380686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:04.381296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:04.381578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:04.381853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:04.391626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:49:04.397013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:49:04.398158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:49:04.399010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 586/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:49:04.403882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:04.506353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6797f-d9b1-472e-8be9-f103b50dfa23 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:04.530464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:04.539162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:04.539765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:04.540044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:04.540315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:04.540974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:49:04.541280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:04.542670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:04.550946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 603/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:49:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:04.558318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:04.591849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:04.653244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1fc2a-07de-4b8c-bd96-3d04d75becca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:04.685308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:04.685933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:04.686221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:04.686491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] PATH_INFO: `/v3/users/3500edb403994da08f5cda3b3f55ab44` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:04.687591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] RBAC: Authorizing `identity:get_user(user_id=3500edb403994da08f5cda3b3f55ab44)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:04.689003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:04.691209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 587/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:49:04 2022] GET /identity/v3/users/3500edb403994da08f5cda3b3f55ab44 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:04.696854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:04.792032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56303-e493-42cd-b63c-213ec2863c24 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:04.824839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e681ee-f757-4e31-9852-344192a13bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:04.825448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e681ee-f757-4e31-9852-344192a13bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:04.825735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e681ee-f757-4e31-9852-344192a13bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:04.826037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e681ee-f757-4e31-9852-344192a13bb9 None admin] PATH_INFO: `/v3/projects/eeccd2f4121e415dba609ae4d2d831ab` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:04.827139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e681ee-f757-4e31-9852-344192a13bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=eeccd2f4121e415dba609ae4d2d831ab)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:04.828566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e681ee-f757-4e31-9852-344192a13bb9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:04.830181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 604/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:49:04 2022] GET /identity/v3/projects/eeccd2f4121e415dba609ae4d2d831ab => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:04.836011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:04.932208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd106a1-ea34-4006-8fb6-ac806a1ba616 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:04.934503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:04.964581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27bb8c-50c1-43e2-9486-9121640d4c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:04.965223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27bb8c-50c1-43e2-9486-9121640d4c80 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:04.965499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27bb8c-50c1-43e2-9486-9121640d4c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:04.965767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27bb8c-50c1-43e2-9486-9121640d4c80 None admin] PATH_INFO: `/v3/projects/eeccd2f4121e415dba609ae4d2d831ab/users/3500edb403994da08f5cda3b3f55ab44/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:04.967652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27bb8c-50c1-43e2-9486-9121640d4c80 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeccd2f4121e415dba609ae4d2d831ab, user_id=3500edb403994da08f5cda3b3f55ab44, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:04.969068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27bb8c-50c1-43e2-9486-9121640d4c80 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:04.981736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 588/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:49:04 2022] PUT /identity/v3/projects/eeccd2f4121e415dba609ae4d2d831ab/users/3500edb403994da08f5cda3b3f55ab44/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:49:07.095178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9fa81-fa9d-412d-994a-192a57297796 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:07.095793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9fa81-fa9d-412d-994a-192a57297796 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:07.095793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9fa81-fa9d-412d-994a-192a57297796 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:07.097716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 605/1193] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:49:07.102988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:07.103372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:07.103536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:07.121366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:49:07.122557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:49:07.185151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 589/1194] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:49:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:49:07.455160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9fa81-fa9d-412d-994a-192a57297796 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:07.511152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:07.553794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9fa81-fa9d-412d-994a-192a57297796 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:07.585773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:07.586411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:07.586692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:07.586965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:07.597172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:49:07.601961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:49:07.603053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:49:07.604084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 606/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:49:07.609020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:07.698170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:07.705564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ff5f8-4c22-4aad-87dc-72aaea1e6732 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:07.737956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:07.738792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:07.739069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:07.739336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:07.739984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:49:07.740281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:07.741622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:07.749304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 590/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:07.755761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:07.850290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0e411-8bdc-4357-9a4d-7554160d0b22 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:07.866343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:07.883964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:07.884573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:07.884853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:07.885127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] PATH_INFO: `/v3/users/3500edb403994da08f5cda3b3f55ab44` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:07.886249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] RBAC: Authorizing `identity:get_user(user_id=3500edb403994da08f5cda3b3f55ab44)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:07.887669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:07.889199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 607/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Sat Sep 24 14:49:07 2022] GET /identity/v3/users/3500edb403994da08f5cda3b3f55ab44 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:07.895016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:07.993255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a1011-6bc1-4c78-8691-3f4afac3ef5e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:08.025113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:08.025711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:08.026025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:08.026305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:08.050558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:08.090858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:08.092283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:08.194036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 591/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:49:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:49:08.200187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:08.297730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33ea26-bc55-48e9-a0ba-226cf83bc57d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:08.329451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a40aa-87c9-45f0-97dc-67b014104a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:08.330088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a40aa-87c9-45f0-97dc-67b014104a64 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:08.330371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a40aa-87c9-45f0-97dc-67b014104a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:08.335841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a40aa-87c9-45f0-97dc-67b014104a64 None admin] PATH_INFO: `/v3/projects/eeccd2f4121e415dba609ae4d2d831ab` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:08.339200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a40aa-87c9-45f0-97dc-67b014104a64 None admin] RBAC: Authorizing `identity:get_project(project_id=eeccd2f4121e415dba609ae4d2d831ab)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:08.340642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a40aa-87c9-45f0-97dc-67b014104a64 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:08.342443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 608/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Sat Sep 24 14:49:08 2022] GET /identity/v3/projects/eeccd2f4121e415dba609ae4d2d831ab => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:08.349033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:49:08.402476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:08.444492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35e607-d17b-45b2-a8c2-6a914a9f3f71 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:49:08.478399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:49:08.479008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:49:08.479290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:49:08.479567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:49:08.480292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=eeccd2f4121e415dba609ae4d2d831ab, user.id=3500edb403994da08f5cda3b3f55ab44) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:49:08.480599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:49:08.482047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0d8d-cc45-4a44-9740-0f0a9e550c8d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:49:08.488663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 592/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:49:08 2022] GET /identity/v3/role_assignments?user.id=3500edb403994da08f5cda3b3f55ab44&scope.project.id=eeccd2f4121e415dba609ae4d2d831ab&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:49:18.325371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=74 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:49:18.485592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=40 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:03.954803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c77ccf-a3d2-40a9-bed8-ecc92073f1c4 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:03.955390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c77ccf-a3d2-40a9-bed8-ecc92073f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:03.955390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c77ccf-a3d2-40a9-bed8-ecc92073f1c4 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:03.957391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 609/1201] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:03.963298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:03.963808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:03.963895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:03.986448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:03.987804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:04.082158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 593/1202] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:04.375361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c77ccf-a3d2-40a9-bed8-ecc92073f1c4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:04.514938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c77ccf-a3d2-40a9-bed8-ecc92073f1c4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:04.574196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:04.574847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:04.575150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:04.575538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:04.589514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:04.590597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:04.610471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:04.612394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:04.613385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:04.613385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:04.613385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:04.613707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 610/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:04.617938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:04.755282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c9db0-ec93-4b9e-98fa-58338cc21e64 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:04.792660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:04.800660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:04.801442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:04.801746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:04.802045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:04.802702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:04.803009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:04.804363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582f75-16f8-4ceb-b14e-9c59afcb365b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:04.821374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 594/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:04.822922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:04.891473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:04.972903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6c8ee-3d7a-4f30-82c0-09e244fc0cba None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:05.017758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97c772-8a0b-4181-8510-9b04996766a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:05.018541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97c772-8a0b-4181-8510-9b04996766a6 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:05.018924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97c772-8a0b-4181-8510-9b04996766a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:05.019198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97c772-8a0b-4181-8510-9b04996766a6 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:05.020070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97c772-8a0b-4181-8510-9b04996766a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:05.021674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97c772-8a0b-4181-8510-9b04996766a6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:05.055337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 611/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Sat Sep 24 14:50:04 2022] POST /identity/v3/users => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:50:07.536703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86fe08-7f47-44d0-8100-decabdf7f2e2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:07.537222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86fe08-7f47-44d0-8100-decabdf7f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:07.537222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86fe08-7f47-44d0-8100-decabdf7f2e2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:07.538807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 595/1206] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:07.544513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:07.544943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:07.545243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:07.568344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:07.569552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:07.654153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 612/1207] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:07.916913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86fe08-7f47-44d0-8100-decabdf7f2e2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:08.027699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86fe08-7f47-44d0-8100-decabdf7f2e2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:08.071058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:08.071714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:08.072197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:08.072484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:08.086890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:50:08.091978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:08.092970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:08.093878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 596/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:08.099143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:08.140483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:08.212987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682380-9078-4c9c-89d3-5de9a75f91ef None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:08.249826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:08.250633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:08.251031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:08.251404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:08.252203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:08.253107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:08.254514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:08.263311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 613/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:08.275016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:08.327926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:08.385921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d89ee-3718-413f-894a-bec8bd11ab0e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:08.420893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:08.421806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:08.422258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:08.422699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:08.437652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:08.439678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:08.457980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:08.459199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:08.460252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:08.460252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:08.460252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:08.467031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 597/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:08.467912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:08.588396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:08.591259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4d850-683f-4a15-aded-996a731d49ed None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:08.628037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:08.628831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:08.629170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:08.629449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:08.630588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:08.631051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:08.632578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:08.646198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 614/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:08.652521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:08.775647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9ca8b-535e-40f3-828e-f82b592c9fb0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:08.777199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:08.814389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:08.815585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:08.815972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:08.816541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:08.825946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:08.827409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:50:08.839020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:08.839985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:08.840963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 598/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:08.846493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:08.969284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c777-a768-4d11-892a-82949d0079d6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:09.007845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:09.008649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:09.009006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:09.009353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:09.010190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:09.010573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:09.012106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:09.031910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 615/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Sat Sep 24 14:50:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:09.039594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:09.158581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51cd3d-2163-4cd0-a8f4-c9d1d237d7fe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:09.202613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:09.203431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:09.204065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:09.204486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:09.237966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:09.284081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:09.285541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:09.403130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 599/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:50:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:50:09.410902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:09.441761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:09.548303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0843b-0593-4377-9f2d-b487dd0e5ee0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:09.592209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:09.592873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:09.593097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:09.593364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:09.608345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:09.610189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:09.631268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:09.632431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:09.633388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:09.633388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:09.633388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:09.633388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 616/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:09.641802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:09.718421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:09.759104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365813f-88cf-4f9f-bf63-4d3462b729ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:09.795446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:09.796058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:09.796343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:09.796619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:09.797272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:09.797582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:09.801626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:09.811403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 600/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:09.818844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:09.890788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:09.923626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2d451-8bf5-4794-956a-9b7775c2e18b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:09.964952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:09.965582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:09.965861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:09.966167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:09.973166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:09.974632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:50:09.986122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:09.987685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:09.994274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 617/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:50:09.994901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:10.102951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bcc31-1a56-4340-af5a-2af4ff963aff None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:10.126573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:10.139164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:10.139775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:10.140059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:10.140334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:10.141032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:10.141343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:10.142804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028b34-f415-4b41-8c19-686699cf16fe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:10.154615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 601/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:10.162612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:10.261743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be5904-e873-4911-a4fc-7922b330df01 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:10.292741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:10.300729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:10.301463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:10.301811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:10.302112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:10.302835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=359a86b84b994530b005b43c6c711884) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:10.303147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:10.304613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153df7d-b521-4893-b98e-37f6120d2676 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:10.310871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 618/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:50:10 2022] GET /identity/v3/role_assignments?user.id=359a86b84b994530b005b43c6c711884&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:12.581754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedd80d-b8e8-482d-b8d5-11613fb1dcde None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:12.582366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedd80d-b8e8-482d-b8d5-11613fb1dcde None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:12.582366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedd80d-b8e8-482d-b8d5-11613fb1dcde None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:12.583692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 602/1220] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:12.589026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:12.589415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:12.589575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:12.608198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:12.609388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:12.689435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 619/1221] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:12.968153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedd80d-b8e8-482d-b8d5-11613fb1dcde None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:13.094214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedd80d-b8e8-482d-b8d5-11613fb1dcde None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:13.132783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:13.133500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:13.133935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:13.134266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:13.145659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:50:13.150986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:13.151961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:13.152846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 603/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:13.160003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:13.264157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1039a-a35f-49a7-9902-16c29ef66708 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:13.288751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:13.305963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:13.305963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:13.305963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:13.305963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:13.306538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:13.306538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:13.307874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:13.317809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 620/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:13.323996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:13.374888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:13.446386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bd932-c874-4307-aab8-d312b08207bb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:13.481836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:13.482493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:13.482774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:13.483048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:13.494788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:13.496309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:13.511741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:13.512923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:13.513814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:13.513814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:13.513814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:13.513814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 604/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:50:13.519042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:13.635752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0469d-65cf-4efb-8631-359c03871cb5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:13.675618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:13.676218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:13.676631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:13.676954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:13.677684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:13.678187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:13.680780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:13.694713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 621/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:13.700776 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:13.803808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:13.811625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025e11-445d-40ab-8749-a4b8bca5d3ba None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:13.845921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:13.847183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:13.847375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:13.847631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:13.855272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:13.856706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:50:13.868622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:13.869603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:13.871158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 605/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:13.876373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:13.930699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:13.988515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba209-ca1b-4213-b7f3-31823d2d4f2a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:14.030306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:14.030907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:14.031322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:14.031784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:14.032907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:14.033395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:14.036132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:14.070035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 622/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Sat Sep 24 14:50:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:50:14.072563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:14.179423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156931a7-9317-4c39-b573-68a3e907dafd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:14.214241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:14.214899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:14.215204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:14.215481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:14.226627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:14.228068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:14.243701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:14.244869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:14.245976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:14.245976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:14.245976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:14.245976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 606/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:14.250800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:14.368407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4b4f2-e0ce-44a0-8709-1664e531e6ef None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:14.371085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:14.406941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:14.407545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:14.407823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:14.408092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:14.409288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:14.409591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:14.410988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:14.422718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 623/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:14.428351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:14.437865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:14.544736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9fa4-9256-4c0a-9c0e-fc62d67f9ba3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:14.583637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:14.584622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:14.584907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:14.585180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:14.592879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:14.594624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:50:14.606859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:14.607996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:14.608933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 607/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:50:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:14.614969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:14.729481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9af38-2f92-46c8-8090-264be110fcea None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:14.763986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:14.764590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:14.764877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:14.765151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:14.765836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:14.766192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:14.767808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695590bc-8284-40b5-b499-e1a1a0404c60 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:14.780995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 624/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:50:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:50:14.787722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:14.856924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:14.897841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a70c9-457f-44d8-9721-0d6d918d8ea6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:14.937314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622179e-a7fd-45b3-9f6c-f628c5390625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:14.938003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622179e-a7fd-45b3-9f6c-f628c5390625 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:14.938288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622179e-a7fd-45b3-9f6c-f628c5390625 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:14.938565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622179e-a7fd-45b3-9f6c-f628c5390625 None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/359a86b84b994530b005b43c6c711884/roles/c8f2545f34b74bb9ab4f1580931915bb` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:14.953926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622179e-a7fd-45b3-9f6c-f628c5390625 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=359a86b84b994530b005b43c6c711884, role_id=c8f2545f34b74bb9ab4f1580931915bb)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:14.955403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622179e-a7fd-45b3-9f6c-f628c5390625 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:14.968981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 608/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:50:14 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/359a86b84b994530b005b43c6c711884/roles/c8f2545f34b74bb9ab4f1580931915bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:50:17.223097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df50cc-84c8-4665-9828-489ce26fb034 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:17.223689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df50cc-84c8-4665-9828-489ce26fb034 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:17.223689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df50cc-84c8-4665-9828-489ce26fb034 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:17.224973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 625/1233] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:50:17.230492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:17.230950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:17.231058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:17.249276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:17.250502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:17.321090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 609/1234] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:50:17.586346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df50cc-84c8-4665-9828-489ce26fb034 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:17.612854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:17.724187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df50cc-84c8-4665-9828-489ce26fb034 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:17.765547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:17.766139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:17.766419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:17.766688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:17.777172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:50:17.783403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:17.784976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:17.785873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 626/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Sat Sep 24 14:50:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:17.791938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:17.844723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:17.909337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dac7e-5cc3-43f1-b56e-892b62ca1a88 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:17.949158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:17.949912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:17.950221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:17.950495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:17.951167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:17.951474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:17.952839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:17.961167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 610/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Sat Sep 24 14:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:17.967859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:18.061036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:18.078850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f83f3-2601-4d36-a829-72627d9636ce None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:18.123165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:18.123740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:18.124021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:18.124292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:18.136855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:18.138892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:18.158014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:18.159779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:18.160867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:18.160867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:18.160867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:18.160867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 627/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:18.167298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:18.283324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bfb8e-decc-4322-bf40-96a443967a91 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:18.294162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:18.323664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:18.324394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:18.324686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:18.325018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:18.325730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:18.326141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:18.327795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:18.341070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 611/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:18.347799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:18.467134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e874-a6bc-42c3-a48c-db74695edcfe None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:18.503088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:18.503642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:18.503922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:18.504186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:18.507240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:18.513136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:18.514674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:50:18.526474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:18.527695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:18.529241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 628/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:18.534348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:18.658040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb7065-cf62-43fe-a181-711d739cd81a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:18.711727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:18.712425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:18.712836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:18.713162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:18.715626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:18.715626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:18.715895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:18.737787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 612/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Sat Sep 24 14:50:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:18.743794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:18.856484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb0f54-21d6-4e01-9a48-ac48acb74fdd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:18.903229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:18.903865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:18.904118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:18.904390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:18.968317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:18.976267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:18.977731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:19.089095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 629/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:50:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:50:19.095026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:19.123263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:19.198627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef7037-b432-4698-b93f-43a04473d2b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:19.237387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:19.238076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:19.238362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:19.238637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:19.249054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:19.250559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:19.265403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:19.266593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:19.267622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:19.267622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:19.267622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:19.267622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 613/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:19.272673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:19.382745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:19.389931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0f723-8090-47de-93b7-88bd43f4e442 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:19.432168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:19.433035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:19.433403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:19.433838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:19.434655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:19.435112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:19.436988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:19.450992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 630/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:19.456698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:19.548505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:19.573017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb3f6-9cdd-4716-bf8c-b2fa5412bc2a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:19.612209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:19.612828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:19.613112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:19.613381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:19.621499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:19.623233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:50:19.645298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:19.646491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:19.650582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 614/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:19.653282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:19.781638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b13af-c538-4c74-ad2f-26529ef568d4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:19.824052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:19.824782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:19.825091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:19.825375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:19.826155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:19.826462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:19.827904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ce602-0cb9-4c65-adf9-12dfd496d9e7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:19.842072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 631/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:19.849093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:19.990037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6ce41-39ff-4785-bd73-f4b587bb4f1b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:20.023505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:20.023983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:20.024264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:20.024539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:20.025259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] RBAC: Adding query filter params (role.id=c8f2545f34b74bb9ab4f1580931915bb, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=359a86b84b994530b005b43c6c711884) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:20.025586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:20.027068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9cebe-b576-439b-b966-08b841139f64 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:20.033035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 615/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:50:19 2022] GET /identity/v3/role_assignments?user.id=359a86b84b994530b005b43c6c711884&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=c8f2545f34b74bb9ab4f1580931915bb => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:22.350496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee62f77-0b38-45b9-9735-349267857b9b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:22.351052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee62f77-0b38-45b9-9735-349267857b9b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:22.351052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee62f77-0b38-45b9-9735-349267857b9b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:22.352555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 632/1247] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:22.357688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:22.358163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:22.358298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:22.366218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:22.379660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:22.380858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:22.454573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 616/1248] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:50:22.722929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee62f77-0b38-45b9-9735-349267857b9b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:22.741880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:22.834447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee62f77-0b38-45b9-9735-349267857b9b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:22.871384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:22.872023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:22.872301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:22.872573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:22.883601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:50:22.888397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:22.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:22.890606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 633/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:22.896167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:23.000198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cc25d-f0d8-4be5-8f67-545e9e956bff None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:23.031200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:23.036895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:23.037501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:23.037785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:23.038095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:23.038759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:23.039061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:23.040426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:23.048917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 617/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:23.055037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:23.136236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:23.160001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d5951-dde3-44d4-83b8-900577ed38bb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:23.192371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:23.192986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:23.193263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:23.193534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:23.203631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:23.205065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:23.219357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:23.220481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:23.221836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:23.221836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:23.221836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:23.221836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 634/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:23.226329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:23.321221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc5fb3-4d1c-4f8e-a311-da05720d3ceb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:23.354034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:23.354632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:23.354918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:23.355194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:23.355836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:23.356147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:23.357500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:23.369107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 618/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:23.374564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:23.474864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0935b8-22d1-4fb6-8520-237f7054675c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:23.500093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:23.508150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e48930-2246-4518-b845-e505927957c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:23.508770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e48930-2246-4518-b845-e505927957c6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:23.509055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e48930-2246-4518-b845-e505927957c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:23.509328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e48930-2246-4518-b845-e505927957c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:23.516615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e48930-2246-4518-b845-e505927957c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:23.518611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e48930-2246-4518-b845-e505927957c6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-94e48930-2246-4518-b845-e505927957c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:50:23.529507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:23.530454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:23.531470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 635/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:23.536921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:23.589972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:23.635168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666598cb-30b0-43ae-98da-a2a8540bb49a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:23.672902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:23.673600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:23.673920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:23.674259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:23.675042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:23.675353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:23.676780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:23.694705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 619/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Sat Sep 24 14:50:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:23.700459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e48930-2246-4518-b845-e505927957c6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:23.796958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e48930-2246-4518-b845-e505927957c6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:23.829212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:23.829798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:23.830115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:23.830942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:23.883193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:23.895393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:23.896825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:23.991359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 636/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:50:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:50:23.997171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:24.092399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5502a-1d00-4ace-a726-3738d704d324 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:24.105750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:24.127442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:24.128088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:24.128368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:24.128642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:24.138445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:24.139826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:24.154569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:24.155730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:24.156815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:24.156815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:24.156815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:24.156815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 620/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:24.161981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:24.249144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:24.261014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdfb08-767a-45e0-b279-5e1d0bcdd263 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:24.294072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:24.294660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:24.294935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:24.295201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:24.295821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:24.296120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:24.297498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:24.308258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 637/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:24.314207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:24.414655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dea39f-158e-45de-93fc-05389dbb3b46 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:24.446240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:24.446844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:24.447122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:24.447394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:24.453695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:24.455144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:50:24.466509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:24.467557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:24.468523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 621/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:24.473665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:24.569741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a8461-f609-42a0-ad4a-e1a531268de3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:24.602437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546884a5-19ba-40ce-a384-866782104907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:24.603032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546884a5-19ba-40ce-a384-866782104907 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:24.603310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546884a5-19ba-40ce-a384-866782104907 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:24.603589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546884a5-19ba-40ce-a384-866782104907 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:24.604269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546884a5-19ba-40ce-a384-866782104907 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:24.604577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546884a5-19ba-40ce-a384-866782104907 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:24.606037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546884a5-19ba-40ce-a384-866782104907 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:24.617846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 638/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:50:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:24.625199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:24.653455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:24.726920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fba63-995f-40cb-a028-7759fa95763e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:24.760417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:24.761005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:24.761284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:24.761555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:24.762301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=359a86b84b994530b005b43c6c711884) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:24.762609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:24.764013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fea96-0cb7-4cf4-8fd0-fa9a8b7f5265 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:24.769184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 622/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:50:24 2022] GET /identity/v3/role_assignments?user.id=359a86b84b994530b005b43c6c711884&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:26.885112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496e540-e015-4274-b90d-d7bf5e1f3f3a None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:26.885660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496e540-e015-4274-b90d-d7bf5e1f3f3a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:26.885742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496e540-e015-4274-b90d-d7bf5e1f3f3a None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:26.887042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 639/1261] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:26.892051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:26.892479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:26.892661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:26.910955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:26.912125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:26.979200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 623/1262] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:27.247637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496e540-e015-4274-b90d-d7bf5e1f3f3a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:27.258459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:27.347555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496e540-e015-4274-b90d-d7bf5e1f3f3a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:27.380207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:27.380817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:27.381092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:27.381357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:27.391055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:50:27.396377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:27.397468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:27.398698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 640/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:50:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:27.403403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:27.421005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:27.498010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ec1a6-7c45-4830-8fc0-ac962665e1fe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:27.531272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:27.531865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:27.532146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:27.532419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:27.533073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:27.533403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:27.534802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:27.542465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 624/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:27.548235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:27.613299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:27.656121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa435c9-a7ba-44e6-a08c-311b53cf2e06 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:27.697748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:27.698398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:27.698673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:27.698940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:27.709191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:27.710653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:27.724664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:27.725764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:27.726699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:27.726699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:27.726699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:27.726699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 641/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:27.731526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:27.809267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:27.828101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af992825-e110-410f-b993-ab72c5f23e23 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:27.859311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:27.859895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:27.860171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:27.860441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:27.861067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:27.861366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:27.863257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:27.874688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 625/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:27.881034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:27.977438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ee085-74ab-425d-b63d-dbf15eb104a0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:27.996777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:28.011340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.011956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.012233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.012503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.019718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.021146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:50:28.031964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:28.033054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:28.034467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 642/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:28.038836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:28.139841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7827a17-f6a5-4d20-8b81-9e2ccef1f262 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:28.172897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.173491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.173773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.174716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.175481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:28.175789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.177221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.185888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:28.194609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 626/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Sat Sep 24 14:50:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:28.200369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:28.294925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b515da-9ff7-4fd8-8a4b-124c5c2e82eb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:28.326660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.327263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.327535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.327800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.337769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.339218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:28.353198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:28.354338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:28.355436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:28.355436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:28.355436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:28.355436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 643/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:28.359871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:28.457728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce914b-8ce6-4d09-b4c0-fed55e401cfa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:28.490784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.491373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.491651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.491922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.492551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:28.492850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.494233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.504876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 627/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:28.510671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:28.541580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:28.608773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9658ef-dcb2-4437-a0cb-c17024b6b71c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:28.644317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.645163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.645548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.645761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.653764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.655246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:50:28.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:28.668465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:28.669579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:28.674050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 644/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:28.676320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:28.734109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:28.774748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57497c-b3a8-41d7-ae72-f7b6bc8dd90d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:28.806858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.807448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.807727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.808002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.808684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:28.808991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.810444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ccb17-38d6-46f1-b730-f1f4811c8bc8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.821849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 628/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:28.827806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:28.916313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:28.925560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c75f4-2d19-4436-9edb-88e65cde0a1b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:28.962888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258faf7e-eed5-4ce4-9068-06f482f25c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:28.963537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258faf7e-eed5-4ce4-9068-06f482f25c83 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:28.963818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258faf7e-eed5-4ce4-9068-06f482f25c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:28.964174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258faf7e-eed5-4ce4-9068-06f482f25c83 None admin] PATH_INFO: `/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/359a86b84b994530b005b43c6c711884/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:28.966073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258faf7e-eed5-4ce4-9068-06f482f25c83 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4a786bafe37463aa9b9eb7bbb2b47b8, user_id=359a86b84b994530b005b43c6c711884, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:28.967509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258faf7e-eed5-4ce4-9068-06f482f25c83 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:28.979717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 645/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Sat Sep 24 14:50:28 2022] PUT /identity/v3/projects/f4a786bafe37463aa9b9eb7bbb2b47b8/users/359a86b84b994530b005b43c6c711884/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:50:31.136185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cff2f7-3b55-4991-937f-5083665361eb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:31.136704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cff2f7-3b55-4991-937f-5083665361eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:31.136704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cff2f7-3b55-4991-937f-5083665361eb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:31.138103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 629/1274] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:31.143855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:31.144267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:31.144406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:31.162218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:31.163390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:31.227166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 646/1275] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:50:31.492449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cff2f7-3b55-4991-937f-5083665361eb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:31.590290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cff2f7-3b55-4991-937f-5083665361eb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:31.606570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:31.623876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:31.624621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:31.624820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:31.625274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:31.637317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:50:31.642834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:31.643784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:31.644635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 630/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Sat Sep 24 14:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:31.650650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:31.740664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:31.759588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5afad9-92e7-4f3f-8290-263b8e253dc9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:31.791595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:31.792195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:31.792475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:31.792746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:31.796844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:31.796844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:31.796844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:31.804363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 647/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Sat Sep 24 14:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:31.810406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:31.907124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d759d-37b6-4b8f-8795-c8f6ff036145 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:31.940498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:31.941120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:31.941401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:31.941674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:31.951469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:31.952933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:31.967034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:31.968150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:31.969223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:31.969223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:31.969223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:31.969223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 631/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:31.974345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:32.073213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442d0a-3abc-437a-b8a1-22e4c1386ad2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:32.105351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:32.105968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:32.106249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:32.106519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:32.107146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:32.107447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:32.108809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:32.120806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 648/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:32.126566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:32.155864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:32.224623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c84fb4-6808-4ae7-81b6-594383de401b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:32.259422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:32.259945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:32.260230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:32.260506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:32.268457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:32.269889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:50:32.280838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:32.282233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:32.284084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 632/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:32.288524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:32.292392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:32.398094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91da080-0e5d-4549-8a25-31afde81ceb0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:32.433273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:32.433870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:32.434179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:32.434450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:32.435678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:32.436224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:32.438288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:32.456263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 649/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Sat Sep 24 14:50:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:32.462104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:32.554716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:32.567878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b79068-6089-49e6-8973-00689c7f5eff None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:32.605702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:32.606646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:32.606933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:32.607209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:32.684812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:32.686207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:32.777283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:32.787590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 633/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Sat Sep 24 14:50:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:50:32.793502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:32.860070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:32.900181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3f2ad-622b-4088-8512-4304eea5a396 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:32.933579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:32.934235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:32.934511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:32.934779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:32.944487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:32.945860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:50:32.959777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:32.960881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:50:32.962155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:50:32.962155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:50:32.962155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:32.962155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 650/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:32.966536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:33.067616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96fc2-ed74-4cdd-9b20-0f3bfa693cbf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:33.101040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:33.101638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:33.101944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:33.102221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:33.102864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:33.103169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:33.104528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:33.115597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 634/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Sat Sep 24 14:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:33.121273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:33.218648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64187f54-4bbd-4023-89ad-60fac218b9f4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:33.232064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:33.251523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:33.252131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:33.252405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:33.252713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:33.259195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:33.260618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:50:33.271960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:33.272975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 24 14:50:33.274274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 651/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Sat Sep 24 14:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:33.278932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:33.327012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:33.375771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb0cf-9e2f-4134-868b-778cd4c2fa40 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:33.408409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:33.409004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:33.409288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:33.409567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:33.410283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:33.410590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:33.412015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559358b-a063-4341-9dff-2c2736771cc7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:33.423592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 635/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Sat Sep 24 14:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:33.429580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:33.526342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad4fc6-5ae3-469c-876c-d4369fc16f68 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:33.557713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:33.558320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:33.558599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:33.558869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:33.559579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] RBAC: Adding query filter params (role.id=7f15657ea9dc4fa986324ff0aa9671ed, scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8, user.id=359a86b84b994530b005b43c6c711884) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:33.559884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:33.561304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba722e8-e884-461d-ad30-81031f65728d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:33.567328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 652/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Sat Sep 24 14:50:33 2022] GET /identity/v3/role_assignments?user.id=359a86b84b994530b005b43c6c711884&scope.project.id=f4a786bafe37463aa9b9eb7bbb2b47b8&role.id=7f15657ea9dc4fa986324ff0aa9671ed => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:35.704083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7765c93-deab-4847-9096-613fa36094cc None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:35.704613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7765c93-deab-4847-9096-613fa36094cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:35.704613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7765c93-deab-4847-9096-613fa36094cc None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:35.706001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 636/1288] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:35.711232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:35.711646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:35.711778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:35.730051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:35.731217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:35.759944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:35.795447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 653/1289] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:36.058108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7765c93-deab-4847-9096-613fa36094cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:36.156145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7765c93-deab-4847-9096-613fa36094cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:36.166954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:36.189307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:36.189970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:36.190256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:36.190532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:36.191111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:36.192469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:36.200991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:36.202219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:36.203165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 637/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:50:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:36.208618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:36.302388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71163-da29-43af-95c0-5a2d632d2672 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:36.334707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:36.335234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:36.335507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:36.335774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:36.336389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] RBAC: Adding query filter params (name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:36.336690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:36.338088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82633e39-1f9e-40ae-b705-6785988151be None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:36.342652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 654/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Sat Sep 24 14:50:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:36.349079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:36.445055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd37925-2165-4f5d-b574-b4adb7cfc256 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:36.477211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:36.477797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:36.478112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:36.478389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:36.479017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:36.479324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:36.480677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af000138-5e44-4f03-86a9-551fbbeb292b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:36.484924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 638/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Sat Sep 24 14:50:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:38.618371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e9d05-d84e-4529-9703-c240eb25ea42 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:38.618933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e9d05-d84e-4529-9703-c240eb25ea42 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:38.618933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e9d05-d84e-4529-9703-c240eb25ea42 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:38.620244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 655/1293] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:38.625796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bd484-b407-48f2-a15f-ff908e264759 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:38.626437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bd484-b407-48f2-a15f-ff908e264759 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:38.626673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bd484-b407-48f2-a15f-ff908e264759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:38.644548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cc2bd484-b407-48f2-a15f-ff908e264759 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:38.645716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc2bd484-b407-48f2-a15f-ff908e264759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:38.675058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:38.711597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 639/1294] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:38.977429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e9d05-d84e-4529-9703-c240eb25ea42 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:39.010547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:39.078785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e9d05-d84e-4529-9703-c240eb25ea42 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:39.112218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c7c0-0d2f-4e18-86f4-795777a57474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:39.112809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c7c0-0d2f-4e18-86f4-795777a57474 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:39.113084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c7c0-0d2f-4e18-86f4-795777a57474 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:39.113349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c7c0-0d2f-4e18-86f4-795777a57474 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:39.114013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c7c0-0d2f-4e18-86f4-795777a57474 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:39.115634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c7c0-0d2f-4e18-86f4-795777a57474 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:39.129390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 656/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Sat Sep 24 14:50:38 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:50:41.248747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b9906-cb91-46f0-a40b-23ecd89d91eb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:41.249282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b9906-cb91-46f0-a40b-23ecd89d91eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:41.249282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b9906-cb91-46f0-a40b-23ecd89d91eb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:41.250739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 640/1296] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:41.256361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:41.256747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:41.256908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:41.274474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:41.275631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:41.344579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 657/1297] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:41.617363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b9906-cb91-46f0-a40b-23ecd89d91eb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:41.725720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b9906-cb91-46f0-a40b-23ecd89d91eb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:41.759329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:41.759945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:41.760230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:41.760504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:41.761089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:41.762478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:41.771576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:41.772707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:41.773798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 641/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:50:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:41.778746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:41.799224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:41.882708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e134ccd-3a30-4b8d-8257-dac45f45e087 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:41.916640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:41.917245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:41.917526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:41.917799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:41.918460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:41.918765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:41.920142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:41.925216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 658/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Sat Sep 24 14:50:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:41.931197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:41.962929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:42.028046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d981d8-fd2b-4e79-ba53-9b37e06a81ea None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:42.059659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:42.060240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:42.060523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:42.060793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:42.061477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] RBAC: Adding query filter params (interface=public, service_id=182f6583e0174a4e8a422f0adc8b4eee, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:42.061780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:42.063156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c560324-fad3-40d6-8e54-f15c9d446b6a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:42.068443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 642/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Sat Sep 24 14:50:41 2022] GET /identity/v3/endpoints?service_id=182f6583e0174a4e8a422f0adc8b4eee&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:50:42.074028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:42.156155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:42.174515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430f96d-fd2d-4bc2-9444-e7f1c7b9d78d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:42.207324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804088d1-ee2d-4833-be69-dd710948fbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:42.207918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804088d1-ee2d-4833-be69-dd710948fbcf None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:42.208198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804088d1-ee2d-4833-be69-dd710948fbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:42.208470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804088d1-ee2d-4833-be69-dd710948fbcf None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:42.209069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804088d1-ee2d-4833-be69-dd710948fbcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:42.210470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804088d1-ee2d-4833-be69-dd710948fbcf None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:42.217088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 659/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Sat Sep 24 14:50:42 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:50:44.330608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98244786-2b62-4959-b3c8-90b3e44765e4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:44.331163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98244786-2b62-4959-b3c8-90b3e44765e4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:44.331163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98244786-2b62-4959-b3c8-90b3e44765e4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:44.332495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 643/1302] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:50:44.337573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:44.338034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:44.338128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:44.356298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:50:44.357453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:44.420679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 660/1303] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:50:44 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:50:44.685756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98244786-2b62-4959-b3c8-90b3e44765e4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:44.778061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:44.784851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98244786-2b62-4959-b3c8-90b3e44765e4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:44.817091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:44.817705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:44.818015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:44.818294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:44.818869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:44.820226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:50:44.828852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:50:44.834075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 24 14:50:44.835228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 644/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Sat Sep 24 14:50:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:50:44.836345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:44.909741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:44.933451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3281a-a471-44b4-aa83-da320b457d32 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:44.966629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:44.967164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:44.967438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:44.967703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:44.968314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:50:44.968612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:44.969992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b8088-4229-40d4-88c8-53a97b67b756 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:44.974842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 661/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Sat Sep 24 14:50:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:50:44.980705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:45.080761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01f66e-0454-46eb-a7e9-0095d3fce9e8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:45.112938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730dc70-430a-4fb6-9c1f-4eedc1505825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:45.113522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730dc70-430a-4fb6-9c1f-4eedc1505825 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:45.113800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730dc70-430a-4fb6-9c1f-4eedc1505825 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:45.114099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730dc70-430a-4fb6-9c1f-4eedc1505825 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:45.114759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730dc70-430a-4fb6-9c1f-4eedc1505825 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:45.116369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730dc70-430a-4fb6-9c1f-4eedc1505825 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:45.135291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 645/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:50:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:50:54.971943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=30 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:55.131148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:48.703867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fbad1-b002-4e3d-8819-1982fd433349 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:48.704919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fbad1-b002-4e3d-8819-1982fd433349 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:48.705135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fbad1-b002-4e3d-8819-1982fd433349 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:48.706675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 662/1307] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:51:48.712296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb849-5087-4455-a814-383bddb7e011 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:48.712719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb849-5087-4455-a814-383bddb7e011 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:48.712934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb849-5087-4455-a814-383bddb7e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:48.859709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fa8fb849-5087-4455-a814-383bddb7e011 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:51:48.861484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa8fb849-5087-4455-a814-383bddb7e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:48.980745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 646/1308] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:51:49.276468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337b3e6-6f32-4daa-a324-fde547a820c8 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:49.276784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337b3e6-6f32-4daa-a324-fde547a820c8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:49.277179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337b3e6-6f32-4daa-a324-fde547a820c8 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:49.278666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 663/1309] 10.222.0.61 () {54 vars in 964 bytes} [Sat Sep 24 14:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:51:49.286550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb018241-1578-410f-a731-d7fb2c42ec3a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:49.286869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb018241-1578-410f-a731-d7fb2c42ec3a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:49.287134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb018241-1578-410f-a731-d7fb2c42ec3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:49.381072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bb018241-1578-410f-a731-d7fb2c42ec3a None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:51:49.382314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bb018241-1578-410f-a731-d7fb2c42ec3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:49.406962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:49.474716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 647/1310] 10.222.0.61 () {58 vars in 1061 bytes} [Sat Sep 24 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:51:49.483595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337b3e6-6f32-4daa-a324-fde547a820c8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:51:49.485327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8337b3e6-6f32-4daa-a324-fde547a820c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:49.597679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337b3e6-6f32-4daa-a324-fde547a820c8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:51:49.679616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:51:49.680232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:49.680512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:49.680786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:49.682409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:49.742795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:49.808628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:51:49.810739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501f41f-2467-4bf7-9a95-306cf4c8c8e2 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:51:49.832634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 664/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:51:52.323378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e37c-b2c5-4ff6-bc29-777b5ca1c7c8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:52.323890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e37c-b2c5-4ff6-bc29-777b5ca1c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:52.323890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e37c-b2c5-4ff6-bc29-777b5ca1c7c8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:52.325221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 648/1312] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:51:52.330374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231342d-bf01-45c3-8084-472865b19680 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:52.330809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231342d-bf01-45c3-8084-472865b19680 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:52.330909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231342d-bf01-45c3-8084-472865b19680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:52.401091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a231342d-bf01-45c3-8084-472865b19680 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:51:52.402317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a231342d-bf01-45c3-8084-472865b19680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:52.498902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 665/1313] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:51:52.779911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e37c-b2c5-4ff6-bc29-777b5ca1c7c8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:51:52.781465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-19c0e37c-b2c5-4ff6-bc29-777b5ca1c7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:52.884411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e37c-b2c5-4ff6-bc29-777b5ca1c7c8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:51:52.958007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:52.960417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:51:52.961060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:52.961523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:52.962038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:52.963885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:53.069832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:51:53.071219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fcae1-2eff-412b-ae07-1a7748dd0509 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:51:53.089336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 649/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:51:55.514491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eda784-0b3b-4c86-82f5-fe3b03352886 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:55.515035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eda784-0b3b-4c86-82f5-fe3b03352886 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:55.515133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eda784-0b3b-4c86-82f5-fe3b03352886 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:55.516479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 666/1315] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:51:55.521889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b46d1-c4f3-4da2-b3fa-432b50fd108f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:55.522353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b46d1-c4f3-4da2-b3fa-432b50fd108f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:55.522485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b46d1-c4f3-4da2-b3fa-432b50fd108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:55.587092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9b3b46d1-c4f3-4da2-b3fa-432b50fd108f None None] MFA Rules not processed for user `2ec8a60b86c745078741d42a19dcb7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:51:55.588295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b3b46d1-c4f3-4da2-b3fa-432b50fd108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:55.639616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:55.683352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 650/1316] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:51:55.966531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eda784-0b3b-4c86-82f5-fe3b03352886 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:51:55.968043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a7eda784-0b3b-4c86-82f5-fe3b03352886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:55.978164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:56.070464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eda784-0b3b-4c86-82f5-fe3b03352886 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:51:56.141308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:51:56.142125 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:56.142410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:56.142684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:56.144265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:56.206428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:56.254248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:51:56.255699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd98127-ed86-4b70-8403-4e6f04b90b99 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:51:56.273231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 667/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:51:58.965620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187bd4-d81e-4b55-8c5d-bb98321adf31 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:58.966145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187bd4-d81e-4b55-8c5d-bb98321adf31 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:58.966244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a187bd4-d81e-4b55-8c5d-bb98321adf31 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:58.967587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 651/1318] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:51:58.973313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659577f9-d95d-4577-9d90-42917bf0bd8b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:58.973718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659577f9-d95d-4577-9d90-42917bf0bd8b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:58.973861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659577f9-d95d-4577-9d90-42917bf0bd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:59.042698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-659577f9-d95d-4577-9d90-42917bf0bd8b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:51:59.043893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-659577f9-d95d-4577-9d90-42917bf0bd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:59.137543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 668/1319] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:51:59.420085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a187bd4-d81e-4b55-8c5d-bb98321adf31 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:51:59.421608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4a187bd4-d81e-4b55-8c5d-bb98321adf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:59.521964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a187bd4-d81e-4b55-8c5d-bb98321adf31 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:51:59.593757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:51:59.594380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:51:59.595201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:51:59.595475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:51:59.597038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:51:59.603874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:51:59.706192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:51:59.707534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a3542-95c3-46bb-95ac-6b9bfe7936a5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:51:59.725256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 652/1320] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:02.507193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539e013-9969-43d4-b57e-5ed432851835 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:02.507749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539e013-9969-43d4-b57e-5ed432851835 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:02.507856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539e013-9969-43d4-b57e-5ed432851835 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:02.509104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 669/1321] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:02.514515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade977b-cdb5-4f41-b9e2-4982f21c865f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:02.515009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade977b-cdb5-4f41-b9e2-4982f21c865f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:02.515089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade977b-cdb5-4f41-b9e2-4982f21c865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:02.589221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ade977b-cdb5-4f41-b9e2-4982f21c865f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:02.590502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ade977b-cdb5-4f41-b9e2-4982f21c865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:02.654977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:02.696349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 653/1322] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:02.992452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e312f-9cdf-4a70-9284-205a20bc7850 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:02.992789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e312f-9cdf-4a70-9284-205a20bc7850 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:02.993013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e312f-9cdf-4a70-9284-205a20bc7850 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:02.994412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 670/1323] 10.222.0.61 () {54 vars in 964 bytes} [Sat Sep 24 14:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:03.000151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c7445-46a5-4ccc-800b-84e6489abd5f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:03.000441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c7445-46a5-4ccc-800b-84e6489abd5f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:03.000741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c7445-46a5-4ccc-800b-84e6489abd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:03.077241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-693c7445-46a5-4ccc-800b-84e6489abd5f None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:03.078480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-693c7445-46a5-4ccc-800b-84e6489abd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:03.153063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 654/1324] 10.222.0.61 () {58 vars in 1061 bytes} [Sat Sep 24 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:03.162512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e312f-9cdf-4a70-9284-205a20bc7850 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:03.164033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-144e312f-9cdf-4a70-9284-205a20bc7850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:03.183657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:03.268395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e312f-9cdf-4a70-9284-205a20bc7850 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:03.341199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:03.341800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:03.342124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:03.342397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:03.343960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:03.418969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:03.458942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:03.460310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368bc09-eca2-4b8b-ade0-33b3d6654882 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:03.477971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 671/1325] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:05.921413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f164f-ed59-4c5c-92f3-c2c2a5ce4c8b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:05.921937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f164f-ed59-4c5c-92f3-c2c2a5ce4c8b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:05.921937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f164f-ed59-4c5c-92f3-c2c2a5ce4c8b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:05.923313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 655/1326] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:05.928540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0af64-2824-40cc-b8a0-7e5250d62596 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:05.928967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0af64-2824-40cc-b8a0-7e5250d62596 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:05.929099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0af64-2824-40cc-b8a0-7e5250d62596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:05.995145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-def0af64-2824-40cc-b8a0-7e5250d62596 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:05.996336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-def0af64-2824-40cc-b8a0-7e5250d62596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:06.095751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 672/1327] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:06.382079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f164f-ed59-4c5c-92f3-c2c2a5ce4c8b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:06.383729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-031f164f-ed59-4c5c-92f3-c2c2a5ce4c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:06.446690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:06.491970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f164f-ed59-4c5c-92f3-c2c2a5ce4c8b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:06.564140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:06.564747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:06.565035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:06.565320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:06.566926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:06.681317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:06.684437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:06.686567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd3cd5-36fa-4f35-9c01-c595aa61c2a8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:06.707194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 656/1328] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:08.918915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c55a41-7a25-485c-8722-2edbb9454fd9 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:08.919454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c55a41-7a25-485c-8722-2edbb9454fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:08.919454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c55a41-7a25-485c-8722-2edbb9454fd9 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:08.922120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 673/1329] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:08.926500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff9ea-b445-4611-bb6b-71192fd06faf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:08.926977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff9ea-b445-4611-bb6b-71192fd06faf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:08.927070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff9ea-b445-4611-bb6b-71192fd06faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:08.999393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0eeff9ea-b445-4611-bb6b-71192fd06faf None None] MFA Rules not processed for user `2ec8a60b86c745078741d42a19dcb7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:09.000622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0eeff9ea-b445-4611-bb6b-71192fd06faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:09.099292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 657/1330] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:09.382697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c55a41-7a25-485c-8722-2edbb9454fd9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:09.384219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30c55a41-7a25-485c-8722-2edbb9454fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:09.415580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:09.490646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c55a41-7a25-485c-8722-2edbb9454fd9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:09.569580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:09.570215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:09.570496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:09.570769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:09.572362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:09.651759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:09.684230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:09.685678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68f8c3-dfd3-40f1-a19a-6bafafa84ea0 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:09.704379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 674/1331] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:12.233277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bc595-f4da-4d52-a61c-85a97eba9c75 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:12.233680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bc595-f4da-4d52-a61c-85a97eba9c75 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:12.233868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bc595-f4da-4d52-a61c-85a97eba9c75 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:12.235802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 658/1332] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:12.241074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b3ff2-eb3a-4065-b806-0a2d43ba1e26 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:12.241726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b3ff2-eb3a-4065-b806-0a2d43ba1e26 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:12.241888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b3ff2-eb3a-4065-b806-0a2d43ba1e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:12.306917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-676b3ff2-eb3a-4065-b806-0a2d43ba1e26 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:12.308118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-676b3ff2-eb3a-4065-b806-0a2d43ba1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:12.401703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 675/1333] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:12.687144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bc595-f4da-4d52-a61c-85a97eba9c75 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:12.688703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c19bc595-f4da-4d52-a61c-85a97eba9c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:12.758388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:12.793182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bc595-f4da-4d52-a61c-85a97eba9c75 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:12.865186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:12.865784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:12.866095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:12.866369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:12.868190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:12.979283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:12.980714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5705647-94fd-4366-abf4-93f0799900d6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:12.990554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:12.999201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 659/1334] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:15.222871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d895a9-db44-441c-aaba-05b4e4ac0301 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:15.223412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d895a9-db44-441c-aaba-05b4e4ac0301 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:15.223412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d895a9-db44-441c-aaba-05b4e4ac0301 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:15.224791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 676/1335] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:15.229915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa72b2-78ea-4cce-b02f-85c757976ec1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:15.230304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa72b2-78ea-4cce-b02f-85c757976ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:15.230494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa72b2-78ea-4cce-b02f-85c757976ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:15.296272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5aaa72b2-78ea-4cce-b02f-85c757976ec1 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:15.298057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5aaa72b2-78ea-4cce-b02f-85c757976ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:15.395476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 660/1336] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:15.681382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d895a9-db44-441c-aaba-05b4e4ac0301 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:15.682953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c8d895a9-db44-441c-aaba-05b4e4ac0301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:15.723600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:15.788308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d895a9-db44-441c-aaba-05b4e4ac0301 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:15.860749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:15.861354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:15.861632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:15.861922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:15.863504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:15.947552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:15.973634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:15.975116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4065e-72e3-45a7-a0d6-175bb05597ba service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:15.992749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 677/1337] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:18.411168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f6cd7-da0d-44c3-b3ad-fda75260736e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:18.411718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f6cd7-da0d-44c3-b3ad-fda75260736e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:18.411718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f6cd7-da0d-44c3-b3ad-fda75260736e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:18.413063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 661/1338] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:18.418178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0e39-b6c5-49b9-b9e0-c3db9fe60410 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:18.418571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0e39-b6c5-49b9-b9e0-c3db9fe60410 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:18.418738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0e39-b6c5-49b9-b9e0-c3db9fe60410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:18.483552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bcde0e39-b6c5-49b9-b9e0-c3db9fe60410 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:18.484660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bcde0e39-b6c5-49b9-b9e0-c3db9fe60410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:18.577382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 678/1339] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:18.867227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f6cd7-da0d-44c3-b3ad-fda75260736e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:18.868751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad9f6cd7-da0d-44c3-b3ad-fda75260736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:18.944574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:18.971217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f6cd7-da0d-44c3-b3ad-fda75260736e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:19.043947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:19.044546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:19.044825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:19.045099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:19.046688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:19.152218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:19.153572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7936c-21e1-4f6d-ac90-c29b2b310d60 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:19.171455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 662/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:23.263239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18732ddd-8cf6-4965-ac48-659de9ce1a1e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:23.263869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18732ddd-8cf6-4965-ac48-659de9ce1a1e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:23.263869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18732ddd-8cf6-4965-ac48-659de9ce1a1e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:23.265056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 679/1341] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:23.270943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb5a3a-412e-4085-933d-9bee1403d6c8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:23.271350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb5a3a-412e-4085-933d-9bee1403d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:23.271631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb5a3a-412e-4085-933d-9bee1403d6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:23.277267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:23.338602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6bcb5a3a-412e-4085-933d-9bee1403d6c8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:23.339714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6bcb5a3a-412e-4085-933d-9bee1403d6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:23.434926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 663/1342] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:23.720170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18732ddd-8cf6-4965-ac48-659de9ce1a1e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:23.721771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-18732ddd-8cf6-4965-ac48-659de9ce1a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:23.769745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:23.824416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18732ddd-8cf6-4965-ac48-659de9ce1a1e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:23.897215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:23.897847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:23.898143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:23.898420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:23.899997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:23.994591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:24.010921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:24.012398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b41197-4430-42ef-bf3f-6a0adb36b411 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:24.030625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 680/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:26.957844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec83597-16a7-4413-ba04-d30dd306784c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:26.958383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec83597-16a7-4413-ba04-d30dd306784c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:26.958477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec83597-16a7-4413-ba04-d30dd306784c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:26.959829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 664/1344] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:26.964990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f415-fadd-4c88-9d6a-f6454e70318c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:26.965422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f415-fadd-4c88-9d6a-f6454e70318c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:26.965548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f415-fadd-4c88-9d6a-f6454e70318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:27.042046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e5a0f415-fadd-4c88-9d6a-f6454e70318c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:27.043262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e5a0f415-fadd-4c88-9d6a-f6454e70318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:27.138745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 681/1345] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:27.423733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec83597-16a7-4413-ba04-d30dd306784c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:27.425263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6ec83597-16a7-4413-ba04-d30dd306784c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:27.518067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:27.535533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec83597-16a7-4413-ba04-d30dd306784c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:27.610654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:27.611258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:27.611533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:27.611799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:27.613365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:27.739178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:27.740735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d187d36-c583-4aa0-b0a1-635842b18fb1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:27.761974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 665/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:31.950804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d31efd-743c-43a9-80de-eac231dc0d86 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:31.951279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d31efd-743c-43a9-80de-eac231dc0d86 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:31.951384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d31efd-743c-43a9-80de-eac231dc0d86 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:31.952683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 682/1347] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:31.958542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f54f20-1825-4b32-869f-e19de1cdce2c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:31.958969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f54f20-1825-4b32-869f-e19de1cdce2c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:31.959090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f54f20-1825-4b32-869f-e19de1cdce2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:31.974257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:32.024869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-54f54f20-1825-4b32-869f-e19de1cdce2c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:32.026107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-54f54f20-1825-4b32-869f-e19de1cdce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:32.120891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 666/1348] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:32.404361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d31efd-743c-43a9-80de-eac231dc0d86 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:32.405885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-07d31efd-743c-43a9-80de-eac231dc0d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:32.467622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:32.516529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d31efd-743c-43a9-80de-eac231dc0d86 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:32.590792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:32.591474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:32.591749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:32.592018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:32.593566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:32.700128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:32.703762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:32.705449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0853d97-d0ea-40fd-8c7e-af49284e1eb4 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:32.723390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 683/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:37.718978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3629ce-00c0-4f05-9a39-ce1ceb24f7aa None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:37.719407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3629ce-00c0-4f05-9a39-ce1ceb24f7aa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:37.719757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3629ce-00c0-4f05-9a39-ce1ceb24f7aa None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:37.721502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 667/1350] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 14:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:37.726797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a66c4-9539-4579-a696-a3d9c9c99575 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:37.727237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a66c4-9539-4579-a696-a3d9c9c99575 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:37.727343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a66c4-9539-4579-a696-a3d9c9c99575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:37.886161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-126a66c4-9539-4579-a696-a3d9c9c99575 None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:37.887355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-126a66c4-9539-4579-a696-a3d9c9c99575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:38.011995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 684/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:42.119235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=59 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:42.165864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcca5ca-4bdc-4de3-ac05-caeec716728b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:42.166447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcca5ca-4bdc-4de3-ac05-caeec716728b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:42.166728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcca5ca-4bdc-4de3-ac05-caeec716728b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:42.168080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 668/1352] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 14:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:42.173083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91481-c268-4b76-8353-632c7e9a0f1c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:42.173528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91481-c268-4b76-8353-632c7e9a0f1c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:42.173696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91481-c268-4b76-8353-632c7e9a0f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:42.232499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:42.259209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-53b91481-c268-4b76-8353-632c7e9a0f1c None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:42.260507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-53b91481-c268-4b76-8353-632c7e9a0f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:42.385151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 685/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:43.695482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4378-1e3d-4614-a6f4-8665eca77f7c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:43.696032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4378-1e3d-4614-a6f4-8665eca77f7c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:43.696032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4378-1e3d-4614-a6f4-8665eca77f7c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:43.697443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 669/1354] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 14:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:43.702520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94ce03-f129-4043-8993-c447cec0b2e7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:43.702950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94ce03-f129-4043-8993-c447cec0b2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:43.703217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94ce03-f129-4043-8993-c447cec0b2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:43.781755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6d94ce03-f129-4043-8993-c447cec0b2e7 None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:43.783564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6d94ce03-f129-4043-8993-c447cec0b2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:43.826630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:43.899060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 686/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:52:47.341799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88893ff2-cc58-430d-b631-ddb2bfb0ebda None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:47.342750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88893ff2-cc58-430d-b631-ddb2bfb0ebda None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:47.343235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88893ff2-cc58-430d-b631-ddb2bfb0ebda None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:47.345145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 670/1356] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:47.350817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a83b7-a0ad-4f45-b851-d337c01946a4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:47.351248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a83b7-a0ad-4f45-b851-d337c01946a4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:47.351449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a83b7-a0ad-4f45-b851-d337c01946a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:47.419976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-974a83b7-a0ad-4f45-b851-d337c01946a4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:47.421170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-974a83b7-a0ad-4f45-b851-d337c01946a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:47.521888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 687/1357] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:47.831179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3cd80-e2cd-48ad-86cc-8325fe291d36 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:47.831469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3cd80-e2cd-48ad-86cc-8325fe291d36 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:47.831746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3cd80-e2cd-48ad-86cc-8325fe291d36 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:47.833221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 671/1358] 10.222.0.61 () {54 vars in 961 bytes} [Sat Sep 24 14:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:47.842309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5340c65-ac73-4f6e-8151-18f9a88dad2a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:47.843566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5340c65-ac73-4f6e-8151-18f9a88dad2a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:47.843987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5340c65-ac73-4f6e-8151-18f9a88dad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:47.866201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:47.936923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a5340c65-ac73-4f6e-8151-18f9a88dad2a None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:47.939065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a5340c65-ac73-4f6e-8151-18f9a88dad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:48.064798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 688/1359] 10.222.0.61 () {58 vars in 1058 bytes} [Sat Sep 24 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:48.072407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3cd80-e2cd-48ad-86cc-8325fe291d36 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:48.073895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-41b3cd80-e2cd-48ad-86cc-8325fe291d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:48.226545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3cd80-e2cd-48ad-86cc-8325fe291d36 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:48.307391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:48.352061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:48.352695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:48.352921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:48.353138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:48.354864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:48.458187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:48.459786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bfcb5-8e96-47b6-ac92-f750d6d3e1e3 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:48.478818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 672/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:50.866664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21fb0-47ec-46fd-b6be-65d6dad652e3 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:50.867452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21fb0-47ec-46fd-b6be-65d6dad652e3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:50.867452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca21fb0-47ec-46fd-b6be-65d6dad652e3 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:50.871625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 689/1361] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:50.874008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cae08-3e8d-4c2e-945f-953535b6040e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:50.874636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cae08-3e8d-4c2e-945f-953535b6040e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:50.874636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cae08-3e8d-4c2e-945f-953535b6040e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:50.956600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c81cae08-3e8d-4c2e-945f-953535b6040e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:50.957803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c81cae08-3e8d-4c2e-945f-953535b6040e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:51.002092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:51.066683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 673/1362] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:51.361742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21fb0-47ec-46fd-b6be-65d6dad652e3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:51.363330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6ca21fb0-47ec-46fd-b6be-65d6dad652e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:51.445649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:51.555848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca21fb0-47ec-46fd-b6be-65d6dad652e3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:51.684642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:51.685293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:51.685592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:51.685853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:51.687503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:51.712539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:51.781893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:51.784290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce946de0-6f1d-4a1f-96b1-93c5f6d3fe36 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:51.802816 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 690/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:52:54.292845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ade051-4922-47e8-8ecd-45124a2dccaa None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:54.293452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ade051-4922-47e8-8ecd-45124a2dccaa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:54.293452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ade051-4922-47e8-8ecd-45124a2dccaa None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:54.294786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 674/1364] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:54.300638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fcaa4-9623-49c2-8d17-8cc473b241bc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:54.301447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fcaa4-9623-49c2-8d17-8cc473b241bc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:54.301727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fcaa4-9623-49c2-8d17-8cc473b241bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:54.371669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c97fcaa4-9623-49c2-8d17-8cc473b241bc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:54.373026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c97fcaa4-9623-49c2-8d17-8cc473b241bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:54.473405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:54.481961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 691/1365] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:54.768666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae184052-4a65-4e84-bc24-0fbc4dc6b65d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:54.768948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae184052-4a65-4e84-bc24-0fbc4dc6b65d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:54.769223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae184052-4a65-4e84-bc24-0fbc4dc6b65d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:54.770786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 675/1366] 10.222.0.61 () {54 vars in 961 bytes} [Sat Sep 24 14:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:54.776345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560ffc2-9acf-43d5-9d75-506a7548aa4c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:54.776637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560ffc2-9acf-43d5-9d75-506a7548aa4c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:54.776916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560ffc2-9acf-43d5-9d75-506a7548aa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:54.858458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9560ffc2-9acf-43d5-9d75-506a7548aa4c None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:54.859654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9560ffc2-9acf-43d5-9d75-506a7548aa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:54.991438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 692/1367] 10.222.0.61 () {58 vars in 1058 bytes} [Sat Sep 24 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:54.999891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae184052-4a65-4e84-bc24-0fbc4dc6b65d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:55.001591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae184052-4a65-4e84-bc24-0fbc4dc6b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:55.157295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae184052-4a65-4e84-bc24-0fbc4dc6b65d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:55.179364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:55.283583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:55.284185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:55.284464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:55.284734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:55.286372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:55.383145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:55.385655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77e2d1-1cfe-4e5a-8d4d-1d268009742f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:55.406367 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 676/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:55.537866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9022f2-524d-4bd5-9fea-cfc623997663 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:55.538425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9022f2-524d-4bd5-9fea-cfc623997663 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:55.538425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9022f2-524d-4bd5-9fea-cfc623997663 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:55.539824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 693/1369] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:55.545894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0318894-1b1a-4742-9ae2-87443dfe3126 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:55.546349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0318894-1b1a-4742-9ae2-87443dfe3126 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:55.546657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0318894-1b1a-4742-9ae2-87443dfe3126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:55.584244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:55.636821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a0318894-1b1a-4742-9ae2-87443dfe3126 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:55.638664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0318894-1b1a-4742-9ae2-87443dfe3126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:55.750108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 677/1370] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:56.091094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456a8f-d50f-4446-a7ec-e549bfbaa2ca None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:56.091094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456a8f-d50f-4446-a7ec-e549bfbaa2ca None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:56.091094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456a8f-d50f-4446-a7ec-e549bfbaa2ca None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:56.092676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 694/1371] 10.222.0.61 () {54 vars in 963 bytes} [Sat Sep 24 14:52:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:56.098306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6864c4-510a-46b5-a18b-e781a9cf09f8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:56.098599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6864c4-510a-46b5-a18b-e781a9cf09f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:56.098874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6864c4-510a-46b5-a18b-e781a9cf09f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:56.180199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2f6864c4-510a-46b5-a18b-e781a9cf09f8 None None] MFA Rules not processed for user `83758a542c2e4314871bf03fd2cca8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:56.181401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2f6864c4-510a-46b5-a18b-e781a9cf09f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:56.231430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:56.270057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 678/1372] 10.222.0.61 () {58 vars in 1060 bytes} [Sat Sep 24 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:56.272974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456a8f-d50f-4446-a7ec-e549bfbaa2ca None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:56.274541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fd456a8f-d50f-4446-a7ec-e549bfbaa2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:56.326311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:56.383279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456a8f-d50f-4446-a7ec-e549bfbaa2ca None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:56.469831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:56.470488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:56.470801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:56.471111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:56.472802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:56.582184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:56.596999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:56.598490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dd246-0a5c-467e-9499-3decaebc7537 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:56.619116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 695/1373] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:52:57.676906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4ebe-e7dd-4810-a225-d5b0785e4dc9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:57.677426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4ebe-e7dd-4810-a225-d5b0785e4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:57.677426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf4ebe-e7dd-4810-a225-d5b0785e4dc9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:57.679052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 679/1374] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:52:57.684468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f65d-de81-4b66-8fbe-762787a15ab3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:57.684941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f65d-de81-4b66-8fbe-762787a15ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:57.685028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f65d-de81-4b66-8fbe-762787a15ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:57.759109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dc38f65d-de81-4b66-8fbe-762787a15ab3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:52:57.760317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc38f65d-de81-4b66-8fbe-762787a15ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:57.856796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 696/1375] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:52:58.154207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf4ebe-e7dd-4810-a225-d5b0785e4dc9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:52:58.155774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-65bf4ebe-e7dd-4810-a225-d5b0785e4dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:58.313863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf4ebe-e7dd-4810-a225-d5b0785e4dc9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:52:58.367024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:52:58.431951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:52:58.432557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:52:58.432834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:52:58.433108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:52:58.434708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:52:58.531565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:52:58.534056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cb142-be90-4b54-9374-eecb9666dd9f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:52:58.554456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 680/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:00.781298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fd305-4f0f-4a67-a64d-bc5ce7256fa9 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:00.781896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fd305-4f0f-4a67-a64d-bc5ce7256fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:00.781896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fd305-4f0f-4a67-a64d-bc5ce7256fa9 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:00.783315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 697/1377] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:00.788274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9f53-a3c6-42be-8966-3914cb52d19f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:00.788686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9f53-a3c6-42be-8966-3914cb52d19f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:00.788825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9f53-a3c6-42be-8966-3914cb52d19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:00.858260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-014d9f53-a3c6-42be-8966-3914cb52d19f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:00.859532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-014d9f53-a3c6-42be-8966-3914cb52d19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:00.869073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:00.971426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 681/1378] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:01.256070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fd305-4f0f-4a67-a64d-bc5ce7256fa9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:01.257642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d90fd305-4f0f-4a67-a64d-bc5ce7256fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:01.321336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:01.413014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fd305-4f0f-4a67-a64d-bc5ce7256fa9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:01.530414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:01.530944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:01.531208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:01.531472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:01.533608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:01.553396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:01.628488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:01.630109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1337b-ea37-4dcd-a565-b6d58977a6b1 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:01.651098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 698/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:03.855613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965caf-79dd-4030-827f-5629d2fa4887 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:03.856081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965caf-79dd-4030-827f-5629d2fa4887 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:03.856165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965caf-79dd-4030-827f-5629d2fa4887 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:03.857681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 682/1380] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:03.863304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b2ddf-ae12-4ea6-9bf4-c1f200c7d2a5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:03.863688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b2ddf-ae12-4ea6-9bf4-c1f200c7d2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:03.863874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b2ddf-ae12-4ea6-9bf4-c1f200c7d2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:03.943724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2e7b2ddf-ae12-4ea6-9bf4-c1f200c7d2a5 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:03.944872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2e7b2ddf-ae12-4ea6-9bf4-c1f200c7d2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:04.038292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:04.058262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 699/1381] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:04.336513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92965caf-79dd-4030-827f-5629d2fa4887 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:04.339202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-92965caf-79dd-4030-827f-5629d2fa4887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:04.515072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:04.527416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92965caf-79dd-4030-827f-5629d2fa4887 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:04.663049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:04.663618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:04.663898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:04.664168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:04.665818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:04.765837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:04.767940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5817be1-02ba-412b-bff9-3d76436a580a service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:04.778026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:04.787153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 683/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:07.042255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e07d3-bd77-4b83-99ba-6883be374f77 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:07.042786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e07d3-bd77-4b83-99ba-6883be374f77 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:07.042786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e07d3-bd77-4b83-99ba-6883be374f77 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:07.044154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 700/1383] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:07.050019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d95b1-fd38-478a-9e10-8563cf65982d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:07.050479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d95b1-fd38-478a-9e10-8563cf65982d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:07.050579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d95b1-fd38-478a-9e10-8563cf65982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:07.118465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-218d95b1-fd38-478a-9e10-8563cf65982d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:07.119718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-218d95b1-fd38-478a-9e10-8563cf65982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:07.226659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 684/1384] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:07.528413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e07d3-bd77-4b83-99ba-6883be374f77 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:07.529984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7d6e07d3-bd77-4b83-99ba-6883be374f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:07.685812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e07d3-bd77-4b83-99ba-6883be374f77 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:07.754630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:07.801738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:07.802299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:07.802586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:07.802855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:07.804439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:07.896767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:07.898400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8bded-fef4-4564-9f22-2d9ec0773c66 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:07.919098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 701/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:10.072614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4823ab8-9e41-418d-86aa-54b83037b6a9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:10.073172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4823ab8-9e41-418d-86aa-54b83037b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:10.073172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4823ab8-9e41-418d-86aa-54b83037b6a9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:10.074516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 685/1386] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:10.079964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7036a4-e234-4a09-80c6-2feaa573d9f6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:10.080418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7036a4-e234-4a09-80c6-2feaa573d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:10.080527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7036a4-e234-4a09-80c6-2feaa573d9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:10.147421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1a7036a4-e234-4a09-80c6-2feaa573d9f6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:10.148566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a7036a4-e234-4a09-80c6-2feaa573d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:10.181821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:10.244965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 702/1387] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:10.534419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4823ab8-9e41-418d-86aa-54b83037b6a9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:10.536015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4823ab8-9e41-418d-86aa-54b83037b6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:10.612899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:10.681354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4823ab8-9e41-418d-86aa-54b83037b6a9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:10.795544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:10.796066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:10.796343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:10.796611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:10.798320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:10.837107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:10.889760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:10.892232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15c445-1b1d-46de-bfcb-53c72e985875 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:10.910204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 686/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:11.832055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ec3d-0446-42dc-9a9e-0c66df35aa60 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:11.832610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ec3d-0446-42dc-9a9e-0c66df35aa60 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:11.832610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ec3d-0446-42dc-9a9e-0c66df35aa60 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:11.834020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 703/1389] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:11.839505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6980-99e8-4b21-9041-ba2d35ac5b47 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:11.839970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6980-99e8-4b21-9041-ba2d35ac5b47 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:11.840075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c6980-99e8-4b21-9041-ba2d35ac5b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:11.906420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e89c6980-99e8-4b21-9041-ba2d35ac5b47 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:11.907646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e89c6980-99e8-4b21-9041-ba2d35ac5b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:12.011533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 687/1390] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:12.313087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5fe3-e526-4ad7-aae9-12ea6c040024 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:12.313391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5fe3-e526-4ad7-aae9-12ea6c040024 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:12.313694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f5fe3-e526-4ad7-aae9-12ea6c040024 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:12.315086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 704/1391] 10.222.0.61 () {54 vars in 962 bytes} [Sat Sep 24 14:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:12.320416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67efbb1-9b9c-4788-8324-c7e3b290e996 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:12.320709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67efbb1-9b9c-4788-8324-c7e3b290e996 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:12.320991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67efbb1-9b9c-4788-8324-c7e3b290e996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:12.337245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:12.397473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c67efbb1-9b9c-4788-8324-c7e3b290e996 None None] MFA Rules not processed for user `09f49af0c95b458291e68167d99199f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:12.398713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c67efbb1-9b9c-4788-8324-c7e3b290e996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:12.480995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 688/1392] 10.222.0.61 () {58 vars in 1059 bytes} [Sat Sep 24 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:12.491093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f5fe3-e526-4ad7-aae9-12ea6c040024 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:12.492841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c84f5fe3-e526-4ad7-aae9-12ea6c040024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:12.596503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f5fe3-e526-4ad7-aae9-12ea6c040024 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:12.655607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:12.671327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:12.671931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:12.672211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:12.673118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:12.674732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:12.784274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:12.785759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5ebe-08e8-4604-be05-ee0278844747 service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:12.803768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 705/1393] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:13.093012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d401231-ebdf-4668-87b0-4dfdc818c4c2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:13.093589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d401231-ebdf-4668-87b0-4dfdc818c4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:13.093589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d401231-ebdf-4668-87b0-4dfdc818c4c2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:13.095504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 689/1394] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:13.100742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:13.101361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:13.101742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:13.169826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:13.171065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:13.217421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:13.279707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 706/1395] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:13.545772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d321ae-13d9-475a-863b-956c1bc52bec None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:13.546153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d321ae-13d9-475a-863b-956c1bc52bec None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:13.546497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d321ae-13d9-475a-863b-956c1bc52bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:13.574620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:13.575832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:13.617847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-19d321ae-13d9-475a-863b-956c1bc52bec None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:13.619107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-19d321ae-13d9-475a-863b-956c1bc52bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:13.680212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:13.711555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8650b-1436-4c89-95e4-4ab5c5702acf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:13.713293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 690/1396] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:13.758700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957c6d-e86e-4de3-acc8-d33df76eab3f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:13.759067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957c6d-e86e-4de3-acc8-d33df76eab3f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:13.759414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957c6d-e86e-4de3-acc8-d33df76eab3f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:13.760850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 691/1397] 10.222.0.61 () {54 vars in 943 bytes} [Sat Sep 24 14:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:13.766988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866d6c2-6d6d-4dbc-84eb-76f8a1a71d1a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:13.767377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866d6c2-6d6d-4dbc-84eb-76f8a1a71d1a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:13.767659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866d6c2-6d6d-4dbc-84eb-76f8a1a71d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:13.771265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:13.831290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:13.831912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:13.832194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:13.832474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:13.834158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:13.848302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6866d6c2-6d6d-4dbc-84eb-76f8a1a71d1a None None] MFA Rules not processed for user `756ac401fea141e68343168d6213d2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:13.849496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6866d6c2-6d6d-4dbc-84eb-76f8a1a71d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:13.925549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:13.927305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:13.946548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 707/1398] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:53:13.964718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 692/1399] 10.222.0.61 () {58 vars in 1040 bytes} [Sat Sep 24 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:53:13.974726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:13.976010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:14.048718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:14.078352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649949-4f4b-459c-9977-de202aa8fd38 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:14.191190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:14.191859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:14.192059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:14.192331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:14.193966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:14.282461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:14.284247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a01a1-a12b-449c-82a4-76928340d716 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:14.292317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 708/1400] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 14:53:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:53:16.186663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38921da1-e3a0-4317-a0ea-fa4632e8b93d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:16.187172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38921da1-e3a0-4317-a0ea-fa4632e8b93d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:16.187172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38921da1-e3a0-4317-a0ea-fa4632e8b93d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:16.188655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 693/1401] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:16.193961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4234c-6be7-4448-b287-64f531a17316 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:16.194449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4234c-6be7-4448-b287-64f531a17316 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:16.194528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4234c-6be7-4448-b287-64f531a17316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:16.209421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:16.266150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-30e4234c-6be7-4448-b287-64f531a17316 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:16.267267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30e4234c-6be7-4448-b287-64f531a17316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:16.379280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 709/1402] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:16.704746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38921da1-e3a0-4317-a0ea-fa4632e8b93d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:16.707471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-38921da1-e3a0-4317-a0ea-fa4632e8b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:16.770306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:16.906817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38921da1-e3a0-4317-a0ea-fa4632e8b93d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:17.025148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:17.035895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:17.038504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:17.038504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:17.038504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:17.038802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:17.140625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:17.142812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c628636-623c-41cb-9659-cc28149686f8 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:17.160824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 694/1403] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:19.359558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8da17-9682-48f5-a5c7-2fe103a08821 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:19.360152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8da17-9682-48f5-a5c7-2fe103a08821 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:19.360152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8da17-9682-48f5-a5c7-2fe103a08821 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:19.361423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 710/1404] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:19.366598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba1cf4-4a23-4032-9612-ebf76bde96ac None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:19.367005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba1cf4-4a23-4032-9612-ebf76bde96ac None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:19.367200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba1cf4-4a23-4032-9612-ebf76bde96ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:19.439304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6eba1cf4-4a23-4032-9612-ebf76bde96ac None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:19.440611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6eba1cf4-4a23-4032-9612-ebf76bde96ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:19.513462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:19.556874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 695/1405] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:19.854655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8da17-9682-48f5-a5c7-2fe103a08821 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:19.856264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-eaa8da17-9682-48f5-a5c7-2fe103a08821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:19.956140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:20.023370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8da17-9682-48f5-a5c7-2fe103a08821 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:20.150792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:20.151315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:20.151592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:20.151863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:20.153477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:20.217717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:20.259503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:20.261427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f92d2-859f-4693-b0f8-c325088f78cd service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:20.284481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 711/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:22.514892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e7a2-316c-4302-97de-479dbf032e92 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:22.516452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e7a2-316c-4302-97de-479dbf032e92 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:22.517063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e7a2-316c-4302-97de-479dbf032e92 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:22.518651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 696/1407] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:53:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:22.524037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173beb80-bfa1-4aa6-b886-e237d9df6fdb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:22.524520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173beb80-bfa1-4aa6-b886-e237d9df6fdb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:22.524625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173beb80-bfa1-4aa6-b886-e237d9df6fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:22.595289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-173beb80-bfa1-4aa6-b886-e237d9df6fdb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:22.596654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-173beb80-bfa1-4aa6-b886-e237d9df6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:22.714586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 712/1408] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:23.013235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e7a2-316c-4302-97de-479dbf032e92 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:23.014923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d52e7a2-316c-4302-97de-479dbf032e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:23.177882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e7a2-316c-4302-97de-479dbf032e92 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:23.232707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:23.303865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:23.304606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:23.304941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:23.305278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:23.306965 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:23.411904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:23.413517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590af16-2ecd-41f5-b96b-85b8466507e9 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:23.432389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 697/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:32.712772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:33.430202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=76 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:36.814543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cf446-bae8-400b-903e-48874e62914f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:36.815114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cf446-bae8-400b-903e-48874e62914f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:36.815114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cf446-bae8-400b-903e-48874e62914f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:36.816413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 713/1410] 10.222.0.61 () {54 vars in 918 bytes} [Sat Sep 24 14:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:36.822068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48176616-5f29-47ba-a0fa-4d0d92f2447c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:36.822532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48176616-5f29-47ba-a0fa-4d0d92f2447c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:36.822833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48176616-5f29-47ba-a0fa-4d0d92f2447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:36.910034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-48176616-5f29-47ba-a0fa-4d0d92f2447c None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:36.911255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-48176616-5f29-47ba-a0fa-4d0d92f2447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:37.015066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 698/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Sat Sep 24 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:53:37.029574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c552-2e9e-4963-9650-edcf3706c704 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:37.029856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c552-2e9e-4963-9650-edcf3706c704 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:37.030154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c552-2e9e-4963-9650-edcf3706c704 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:37.031469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 714/1412] 10.222.0.61 () {54 vars in 918 bytes} [Sat Sep 24 14:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:37.036660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d01-703c-4a07-a25d-5567a951d19c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:37.036947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d01-703c-4a07-a25d-5567a951d19c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:37.037218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d01-703c-4a07-a25d-5567a951d19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:37.109298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:37.112581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-75cc7d01-703c-4a07-a25d-5567a951d19c None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:37.114532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-75cc7d01-703c-4a07-a25d-5567a951d19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:37.227790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 699/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Sat Sep 24 14:53:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:39.757827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc018-6006-42cd-8e7b-1ca4ab0228f2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:39.758418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc018-6006-42cd-8e7b-1ca4ab0228f2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:39.758528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc018-6006-42cd-8e7b-1ca4ab0228f2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:39.759888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:53:39.765228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844af6af-c553-4e50-b0c9-d323db7d57f1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:39.765651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844af6af-c553-4e50-b0c9-d323db7d57f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:39.765781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844af6af-c553-4e50-b0c9-d323db7d57f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:39.832529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-844af6af-c553-4e50-b0c9-d323db7d57f1 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:39.833738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-844af6af-c553-4e50-b0c9-d323db7d57f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:39.888873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:39.932540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:40.221834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc018-6006-42cd-8e7b-1ca4ab0228f2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:40.223389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8d4bc018-6006-42cd-8e7b-1ca4ab0228f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:40.366356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc018-6006-42cd-8e7b-1ca4ab0228f2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:40.439283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:40.478540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:40.479143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:40.479419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:40.479687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:40.481270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:40.570550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:40.572136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d514f6-e1bd-488c-bf5f-29ed7120e42f service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:40.589493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 716/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:41.720296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8e679-7f9b-44a9-b736-2060babe6880 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:41.720867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8e679-7f9b-44a9-b736-2060babe6880 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:41.720867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8e679-7f9b-44a9-b736-2060babe6880 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:41.722244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:41.728425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b4e9-ade0-482f-b471-a889f0f57215 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:41.728829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b4e9-ade0-482f-b471-a889f0f57215 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:41.728968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b4e9-ade0-482f-b471-a889f0f57215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:41.806767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1991b4e9-ade0-482f-b471-a889f0f57215 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:41.807977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1991b4e9-ade0-482f-b471-a889f0f57215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:41.835458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:41.906497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:42.056910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8e679-7f9b-44a9-b736-2060babe6880 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:42.058424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7bf8e679-7f9b-44a9-b736-2060babe6880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:42.202332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8e679-7f9b-44a9-b736-2060babe6880 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:42.243715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:42.316796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:42.317342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:42.317623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:42.317897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:42.319519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:42.407043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:42.408638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbb6b2-7f13-4742-b363-e5f8c37bd338 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:42.426750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 702/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:43.549397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24222de-df64-4355-99c6-454d10169da2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:43.549969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24222de-df64-4355-99c6-454d10169da2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:43.549969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24222de-df64-4355-99c6-454d10169da2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:43.551295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:43.557634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6798ae-ca74-4fa2-8bfd-fbe086e93772 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:43.558120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6798ae-ca74-4fa2-8bfd-fbe086e93772 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:43.558226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6798ae-ca74-4fa2-8bfd-fbe086e93772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:43.612070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:43.634162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8b6798ae-ca74-4fa2-8bfd-fbe086e93772 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:43.635366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b6798ae-ca74-4fa2-8bfd-fbe086e93772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:43.730417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:43.885966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24222de-df64-4355-99c6-454d10169da2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:43.887410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b24222de-df64-4355-99c6-454d10169da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:44.029624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24222de-df64-4355-99c6-454d10169da2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:44.041667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:44.142045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:44.142525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:44.142807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:44.143083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:44.144693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:44.236371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:44.237991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419443-6db1-4807-bb0f-0dde85d1919e service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:44.256208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 719/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:45.360635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322007-8dc1-4c68-beec-00903d3374d7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:45.361195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322007-8dc1-4c68-beec-00903d3374d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:45.361306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322007-8dc1-4c68-beec-00903d3374d7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:45.362645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:45.369819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2b63-d83f-4ef4-87ed-1616db071669 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:45.370307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2b63-d83f-4ef4-87ed-1616db071669 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:45.370398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2b63-d83f-4ef4-87ed-1616db071669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:45.391828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:45.446469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-616d2b63-d83f-4ef4-87ed-1616db071669 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:45.447678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-616d2b63-d83f-4ef4-87ed-1616db071669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:45.542054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:45.690688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59322007-8dc1-4c68-beec-00903d3374d7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:45.692196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-59322007-8dc1-4c68-beec-00903d3374d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:45.803064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:45.838808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59322007-8dc1-4c68-beec-00903d3374d7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:45.954144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:45.954827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:45.955111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:45.955384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:45.956988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:46.036424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:46.049952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:46.051559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079676cf-cdcc-4179-8aaa-ab3c3b50c475 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:46.069761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 705/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:47.182637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3aa29-ed18-48e6-9581-a58a459df519 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:47.183138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3aa29-ed18-48e6-9581-a58a459df519 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:47.183138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3aa29-ed18-48e6-9581-a58a459df519 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:47.184452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:47.190907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f0959-0bb7-456c-a384-a3a6effd39e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:47.191390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f0959-0bb7-456c-a384-a3a6effd39e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:47.191492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f0959-0bb7-456c-a384-a3a6effd39e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:47.269182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a78f0959-0bb7-456c-a384-a3a6effd39e9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:47.270455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a78f0959-0bb7-456c-a384-a3a6effd39e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:47.365754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:47.513544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3aa29-ed18-48e6-9581-a58a459df519 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:47.515107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bea3aa29-ed18-48e6-9581-a58a459df519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:47.598316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:47.659689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3aa29-ed18-48e6-9581-a58a459df519 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:47.777779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:47.778340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:47.778619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:47.778890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:47.780491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:47.835360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:47.876250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:47.877844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e6a7e-1dbf-4470-9bff-2e50226e8ed5 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:47.895904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 722/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:48.930902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e449f-c31e-40ca-bb98-03c4df3060ac None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:48.931456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e449f-c31e-40ca-bb98-03c4df3060ac None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:48.931456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e449f-c31e-40ca-bb98-03c4df3060ac None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:48.932819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:48.939496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf438a-c092-43bd-be1b-ab1fdcc287f2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:48.939954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf438a-c092-43bd-be1b-ab1fdcc287f2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:48.940056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf438a-c092-43bd-be1b-ab1fdcc287f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:49.014460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-efdf438a-c092-43bd-be1b-ab1fdcc287f2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:49.016153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-efdf438a-c092-43bd-be1b-ab1fdcc287f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:49.110378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:49.259693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e449f-c31e-40ca-bb98-03c4df3060ac None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:49.261228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c91e449f-c31e-40ca-bb98-03c4df3060ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:49.310819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:49.407591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e449f-c31e-40ca-bb98-03c4df3060ac None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:49.520741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:49.521341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:49.521626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:49.521938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:49.523522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:49.530449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:49.612686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:49.614345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf63e90-bd67-4dc8-8aab-c17742b35c1f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:49.631979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 708/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:50.693653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb69a5-e434-46b5-9041-57c3050724a2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:50.694216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb69a5-e434-46b5-9041-57c3050724a2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:50.694216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb69a5-e434-46b5-9041-57c3050724a2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:50.695493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:50.701628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bc4a8-12e2-4a06-83fd-bbca327ff4c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:50.702067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bc4a8-12e2-4a06-83fd-bbca327ff4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:50.702205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bc4a8-12e2-4a06-83fd-bbca327ff4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:50.777748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a82bc4a8-12e2-4a06-83fd-bbca327ff4c5 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:50.778989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a82bc4a8-12e2-4a06-83fd-bbca327ff4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:50.844807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:50.873722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:53:51.023725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb69a5-e434-46b5-9041-57c3050724a2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:51.025241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30bb69a5-e434-46b5-9041-57c3050724a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:51.042519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:51.169666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb69a5-e434-46b5-9041-57c3050724a2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:51.255518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:51.280885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:51.281957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:51.282241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:51.282510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:51.284078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:51.371630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:51.373216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7fe36-a257-4b70-93e0-758340f9329b service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:51.390687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 725/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:52.440759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b2327-a867-484f-a060-62dd0e5b9c74 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:52.441312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b2327-a867-484f-a060-62dd0e5b9c74 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:52.441312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b2327-a867-484f-a060-62dd0e5b9c74 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:52.442716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:52.448826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a14598-dff4-416e-ad5a-e92d70b71edc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:52.449292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a14598-dff4-416e-ad5a-e92d70b71edc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:52.449393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a14598-dff4-416e-ad5a-e92d70b71edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:52.523827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a4a14598-dff4-416e-ad5a-e92d70b71edc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:52.525016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4a14598-dff4-416e-ad5a-e92d70b71edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:52.566354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:52.620593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:52.769432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b2327-a867-484f-a060-62dd0e5b9c74 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:52.770957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9c7b2327-a867-484f-a060-62dd0e5b9c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:52.916052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b2327-a867-484f-a060-62dd0e5b9c74 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:52.970556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:53.028906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:53.029511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:53.029796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:53.030101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:53.031687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:53.119592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:53.121171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fbaa35-c140-4347-a3c4-a385aaa988de service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:53.138840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 711/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:54.180999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c79c-199c-4df8-b0da-5ba22e05974d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:54.181573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c79c-199c-4df8-b0da-5ba22e05974d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:54.181573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c79c-199c-4df8-b0da-5ba22e05974d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:54.183002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:54.189143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13135a7-0673-4390-858b-42cb953f3c5b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:54.189595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13135a7-0673-4390-858b-42cb953f3c5b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:54.189703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13135a7-0673-4390-858b-42cb953f3c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:54.264576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a13135a7-0673-4390-858b-42cb953f3c5b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:54.266312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a13135a7-0673-4390-858b-42cb953f3c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:54.268284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:54.363705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:54.511325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962c79c-199c-4df8-b0da-5ba22e05974d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:54.512838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3962c79c-199c-4df8-b0da-5ba22e05974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:54.656442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962c79c-199c-4df8-b0da-5ba22e05974d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:54.684581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:54.770910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:54.771431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:54.771703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:54.771968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:54.773570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:54.860204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:54.861796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3b9c-40bb-410c-9401-c4169df3853a service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:54.880257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 728/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:56.953333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eedf0-c904-44f2-aced-d359f38f8388 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:56.953822 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eedf0-c904-44f2-aced-d359f38f8388 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:56.953968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88eedf0-c904-44f2-aced-d359f38f8388 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:56.955253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:56.960646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4fa9-3ca6-4644-beb8-2e5e49252dc4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:56.961059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4fa9-3ca6-4644-beb8-2e5e49252dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:56.961202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4fa9-3ca6-4644-beb8-2e5e49252dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:57.001726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:57.027253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e13c4fa9-3ca6-4644-beb8-2e5e49252dc4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:53:57.028468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e13c4fa9-3ca6-4644-beb8-2e5e49252dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:57.123276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:53:57.413330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eedf0-c904-44f2-aced-d359f38f8388 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:53:57.414911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c88eedf0-c904-44f2-aced-d359f38f8388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:57.519396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88eedf0-c904-44f2-aced-d359f38f8388 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:53:57.551697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:53:57.593929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:53:57.594541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:57.594824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:57.595095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:57.596677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:53:57.704763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:53:57.706144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061a494-41e0-4ad1-a05b-8f4dc3dca2d9 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:53:57.725351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 714/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:53:59.987230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1467b2a-5c7b-4ec1-acf8-f18a2abbcc7f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:59.987753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1467b2a-5c7b-4ec1-acf8-f18a2abbcc7f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:59.987863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1467b2a-5c7b-4ec1-acf8-f18a2abbcc7f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:53:59.989139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:53:59.994501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92f73a-8dd6-4f4f-b2c4-b9f93ba2c94f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:53:59.994943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92f73a-8dd6-4f4f-b2c4-b9f93ba2c94f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:53:59.995046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92f73a-8dd6-4f4f-b2c4-b9f93ba2c94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:00.059483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5b92f73a-8dd6-4f4f-b2c4-b9f93ba2c94f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:00.060691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5b92f73a-8dd6-4f4f-b2c4-b9f93ba2c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:00.073942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:00.155439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:00.432859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1467b2a-5c7b-4ec1-acf8-f18a2abbcc7f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:00.434445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b1467b2a-5c7b-4ec1-acf8-f18a2abbcc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:00.538683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1467b2a-5c7b-4ec1-acf8-f18a2abbcc7f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:00.551460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:00.612081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:00.612680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:00.612960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:00.613229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:00.614833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:00.723032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:00.724459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ca59-5039-4ed6-ba6f-ff76bf06bbf2 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:00.742770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 731/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:03.154083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82da192-ab26-43c9-846d-97f9773ddaf4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:03.154637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82da192-ab26-43c9-846d-97f9773ddaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:03.154637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82da192-ab26-43c9-846d-97f9773ddaf4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:03.156035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:03.161292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9121-0d82-49ac-b287-e2d7b6476747 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:03.161725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9121-0d82-49ac-b287-e2d7b6476747 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:03.161853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9121-0d82-49ac-b287-e2d7b6476747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:03.209977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:03.227703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-356b9121-0d82-49ac-b287-e2d7b6476747 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:03.228907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-356b9121-0d82-49ac-b287-e2d7b6476747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:03.325308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:03.604516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82da192-ab26-43c9-846d-97f9773ddaf4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:03.606084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b82da192-ab26-43c9-846d-97f9773ddaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:03.708463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82da192-ab26-43c9-846d-97f9773ddaf4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:03.748108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:03.783157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:03.783757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:03.784037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:03.784311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:03.785885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:03.893488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:03.894866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc31319-60ab-4c62-8fed-3c827bff52f8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:03.913072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 717/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:06.174982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112ae28-d42b-4dbf-be04-59dd8cbfc4c0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:06.175566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112ae28-d42b-4dbf-be04-59dd8cbfc4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:06.175566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112ae28-d42b-4dbf-be04-59dd8cbfc4c0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:06.176870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:06.182176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f281-1f46-4c45-aec2-a2d26bb9a0ae None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:06.182583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f281-1f46-4c45-aec2-a2d26bb9a0ae None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:06.182698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f281-1f46-4c45-aec2-a2d26bb9a0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:06.250389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4dd8f281-1f46-4c45-aec2-a2d26bb9a0ae None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:06.251615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4dd8f281-1f46-4c45-aec2-a2d26bb9a0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:06.272103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:06.348030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:54:06.631059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112ae28-d42b-4dbf-be04-59dd8cbfc4c0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:06.632623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4112ae28-d42b-4dbf-be04-59dd8cbfc4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:06.734949 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112ae28-d42b-4dbf-be04-59dd8cbfc4c0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:06.755774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:06.808338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:06.808963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:06.809265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:06.809562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:06.811166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:06.917794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:06.919190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8125e1d-ce28-4670-b87f-03aea8115d55 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:06.938272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 734/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:09.136956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252df9a-ff75-4e90-b9c4-c81fc38f867b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:09.137515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252df9a-ff75-4e90-b9c4-c81fc38f867b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:09.137515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252df9a-ff75-4e90-b9c4-c81fc38f867b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:09.138994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:09.144859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd007-3571-400c-a80f-f0aea2f1b19f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:09.145289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd007-3571-400c-a80f-f0aea2f1b19f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:09.145401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd007-3571-400c-a80f-f0aea2f1b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:09.207835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:09.210408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f48bd007-3571-400c-a80f-f0aea2f1b19f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:09.212194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f48bd007-3571-400c-a80f-f0aea2f1b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:09.308930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:54:09.592409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252df9a-ff75-4e90-b9c4-c81fc38f867b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:09.594033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3252df9a-ff75-4e90-b9c4-c81fc38f867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:09.696768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252df9a-ff75-4e90-b9c4-c81fc38f867b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:09.744647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:09.768810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:09.769403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:09.769707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:09.770000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:09.771559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:09.877952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:09.879328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760df49-2aba-42ec-8fa8-8bef887503b1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:09.898937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 720/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:12.065556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d7068-2d3a-46c2-adee-6684a06b59dd None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:12.066091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d7068-2d3a-46c2-adee-6684a06b59dd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:12.066198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d7068-2d3a-46c2-adee-6684a06b59dd None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:12.067450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:12.072733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9de65-c13a-485c-abf4-4c66151df9db None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:12.073131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9de65-c13a-485c-abf4-4c66151df9db None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:12.073407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9de65-c13a-485c-abf4-4c66151df9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:12.140380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a9d9de65-c13a-485c-abf4-4c66151df9db None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:12.141564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a9d9de65-c13a-485c-abf4-4c66151df9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:12.292434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:12.360278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:12.641861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d7068-2d3a-46c2-adee-6684a06b59dd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:12.643438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e63d7068-2d3a-46c2-adee-6684a06b59dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:12.744712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d7068-2d3a-46c2-adee-6684a06b59dd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:12.773467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:12.819629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:12.820228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:12.820509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:12.820778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:12.822374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:12.931387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:12.932752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78705a-5ce0-401a-9db0-53729ac7863b service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:12.951640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 737/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:15.129231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366529ea-4a3c-4b88-b201-14b6f5633aa5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:15.129757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366529ea-4a3c-4b88-b201-14b6f5633aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:15.129757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366529ea-4a3c-4b88-b201-14b6f5633aa5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:15.131160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:15.136611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312aef1-e016-4960-97b9-f0624c20fd36 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:15.137058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312aef1-e016-4960-97b9-f0624c20fd36 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:15.137158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312aef1-e016-4960-97b9-f0624c20fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:15.201361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d312aef1-e016-4960-97b9-f0624c20fd36 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:15.202908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d312aef1-e016-4960-97b9-f0624c20fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:15.210995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:15.298214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:15.597987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366529ea-4a3c-4b88-b201-14b6f5633aa5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:15.599525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-366529ea-4a3c-4b88-b201-14b6f5633aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:15.704112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366529ea-4a3c-4b88-b201-14b6f5633aa5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:15.759090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:15.777038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:15.777640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:15.777947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:15.778224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:15.780424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:15.886412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:15.887770 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8ad4c-6179-4f26-8648-07cd1f41da55 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:15.907193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 723/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:18.086734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d460aa-f406-4b69-a62c-3bb222cecd03 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:18.087306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d460aa-f406-4b69-a62c-3bb222cecd03 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:18.087306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d460aa-f406-4b69-a62c-3bb222cecd03 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:18.088603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:18.094556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7442a-42ca-4915-817f-b1b971cc2b7c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:18.094956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7442a-42ca-4915-817f-b1b971cc2b7c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:18.095121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7442a-42ca-4915-817f-b1b971cc2b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:18.161617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3ad7442a-42ca-4915-817f-b1b971cc2b7c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:18.162875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3ad7442a-42ca-4915-817f-b1b971cc2b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:18.199735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:18.257808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:18.545634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d460aa-f406-4b69-a62c-3bb222cecd03 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:18.547226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-23d460aa-f406-4b69-a62c-3bb222cecd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:18.649212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d460aa-f406-4b69-a62c-3bb222cecd03 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:18.689573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:18.725753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:18.726384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:18.726657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:18.726924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:18.728487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:18.836798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:18.838295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8beda-dc6f-4bd3-89c4-df84fb12d724 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:18.856198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 740/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:21.020570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152088-ea3b-477e-be9a-c5918fd9ee6b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:21.021125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152088-ea3b-477e-be9a-c5918fd9ee6b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:21.021125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152088-ea3b-477e-be9a-c5918fd9ee6b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:21.022529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:21.027853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983e61-272c-418e-b2af-96404903174a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:21.028290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983e61-272c-418e-b2af-96404903174a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:21.028426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983e61-272c-418e-b2af-96404903174a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:21.092465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-53983e61-272c-418e-b2af-96404903174a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:21.093647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-53983e61-272c-418e-b2af-96404903174a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:21.111770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:21.189805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:21.471271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152088-ea3b-477e-be9a-c5918fd9ee6b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:21.472793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6152088-ea3b-477e-be9a-c5918fd9ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:21.574945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152088-ea3b-477e-be9a-c5918fd9ee6b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:21.640968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:21.648417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:21.649020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:21.649305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:21.649585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:21.651217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:21.762365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:21.764323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f8cbd-01b7-41b5-9677-de6817e08b41 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:21.782747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 726/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:23.941105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b00030-8485-4b19-bbf1-c71d1804c2e9 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:23.941537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b00030-8485-4b19-bbf1-c71d1804c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:23.941644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b00030-8485-4b19-bbf1-c71d1804c2e9 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:23.943027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:23.948274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aa9de-f4a0-466f-bdd4-90ba9a1efe13 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:23.948713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aa9de-f4a0-466f-bdd4-90ba9a1efe13 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:23.948828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aa9de-f4a0-466f-bdd4-90ba9a1efe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:24.015644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c44aa9de-f4a0-466f-bdd4-90ba9a1efe13 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:24.016869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c44aa9de-f4a0-466f-bdd4-90ba9a1efe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:24.064943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:24.113838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:54:24.401114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b00030-8485-4b19-bbf1-c71d1804c2e9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:24.402691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f2b00030-8485-4b19-bbf1-c71d1804c2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:24.505049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b00030-8485-4b19-bbf1-c71d1804c2e9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:24.552302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:24.578415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:24.578961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:24.579243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:24.579517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:24.581091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:24.690589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:24.692041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fab657-66e4-418d-a23b-73558ef11e4d service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:24.709575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 743/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:34.112298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=22 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:34.707433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=66 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:44.570740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4598b-995b-456a-8bb5-3c1379deed83 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:44.571269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4598b-995b-456a-8bb5-3c1379deed83 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:44.571269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4598b-995b-456a-8bb5-3c1379deed83 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:44.572626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 728/1471] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:44.578726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b94fa-ed1c-4d41-a3cb-595e226f2650 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:44.579143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b94fa-ed1c-4d41-a3cb-595e226f2650 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:44.579283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b94fa-ed1c-4d41-a3cb-595e226f2650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:44.649175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a23b94fa-ed1c-4d41-a3cb-595e226f2650 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:44.650419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a23b94fa-ed1c-4d41-a3cb-595e226f2650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:44.744513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 744/1472] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:45.047130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64a2d-1365-4057-8658-c5fe96a62c44 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:45.047417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64a2d-1365-4057-8658-c5fe96a62c44 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:45.047693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64a2d-1365-4057-8658-c5fe96a62c44 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:45.049067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 729/1473] 10.222.0.61 () {54 vars in 962 bytes} [Sat Sep 24 14:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:45.054189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec5d5-7bb9-4921-a46e-852f36780b0c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:45.054474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec5d5-7bb9-4921-a46e-852f36780b0c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:45.054747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ec5d5-7bb9-4921-a46e-852f36780b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:45.129477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ff3ec5d5-7bb9-4921-a46e-852f36780b0c None None] MFA Rules not processed for user `09f49af0c95b458291e68167d99199f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:45.130749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ff3ec5d5-7bb9-4921-a46e-852f36780b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:45.151505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:45.206753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 745/1474] 10.222.0.61 () {58 vars in 1059 bytes} [Sat Sep 24 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:45.214590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a64a2d-1365-4057-8658-c5fe96a62c44 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:45.216091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-39a64a2d-1365-4057-8658-c5fe96a62c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:45.318645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a64a2d-1365-4057-8658-c5fe96a62c44 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:45.392816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:45.393409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:45.393691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:45.393997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:45.395574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:45.449130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:45.503266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:45.504690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f9afa-7159-4b87-a0dd-77ec9f4069d2 service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:45.523030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 730/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:47.787575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1bd1b-eee1-4e8c-932b-17846cb84272 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:47.788141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1bd1b-eee1-4e8c-932b-17846cb84272 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:47.788141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1bd1b-eee1-4e8c-932b-17846cb84272 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:47.789648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 746/1476] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:54:47.794768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76597-6441-49c7-93b0-7123c0f2b52e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:47.795169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76597-6441-49c7-93b0-7123c0f2b52e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:47.795324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76597-6441-49c7-93b0-7123c0f2b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:47.859627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e2b76597-6441-49c7-93b0-7123c0f2b52e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:47.860818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2b76597-6441-49c7-93b0-7123c0f2b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:47.960401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 731/1477] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:48.243211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1bd1b-eee1-4e8c-932b-17846cb84272 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:48.244824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fef1bd1b-eee1-4e8c-932b-17846cb84272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:48.372746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1bd1b-eee1-4e8c-932b-17846cb84272 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:48.423505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:48.466253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:48.466883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:48.467157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:48.467442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:48.468087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:48.469840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:54:48.469840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:54:48.469840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:54:48.471008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69acceb-7ff9-4792-a8cb-5c18e9512fcb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:48.490296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 747/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Sat Sep 24 14:54:48 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:54:50.656617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59fce-367b-4ca4-b5d7-2f6d1041ad28 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:50.657172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59fce-367b-4ca4-b5d7-2f6d1041ad28 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:50.657172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59fce-367b-4ca4-b5d7-2f6d1041ad28 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:50.658498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 732/1479] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:50.663666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7a414-265e-4ec2-9eb8-99462e48ff57 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:50.664072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7a414-265e-4ec2-9eb8-99462e48ff57 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:50.664214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7a414-265e-4ec2-9eb8-99462e48ff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:50.728753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-11b7a414-265e-4ec2-9eb8-99462e48ff57 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:50.729952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-11b7a414-265e-4ec2-9eb8-99462e48ff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:50.825712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 748/1480] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:51.120782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59fce-367b-4ca4-b5d7-2f6d1041ad28 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:51.122323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d0f59fce-367b-4ca4-b5d7-2f6d1041ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:51.132006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:51.265751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59fce-367b-4ca4-b5d7-2f6d1041ad28 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:51.344800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:51.378151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:51.378749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:51.379024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:51.379290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:51.380861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:51.468389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:51.470011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb620e-726f-47fc-baa7-8364496ac14e service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:51.487888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 733/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:53.801599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aecd4e-fd68-4d53-9483-5fe4eb39b314 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:53.802212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aecd4e-fd68-4d53-9483-5fe4eb39b314 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:53.802212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aecd4e-fd68-4d53-9483-5fe4eb39b314 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:53.803513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 749/1482] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:53.808736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c627a3-3b23-45a2-810e-c4b5ca79efc5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:53.809171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c627a3-3b23-45a2-810e-c4b5ca79efc5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:53.809280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c627a3-3b23-45a2-810e-c4b5ca79efc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:53.874900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-26c627a3-3b23-45a2-810e-c4b5ca79efc5 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:53.876099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26c627a3-3b23-45a2-810e-c4b5ca79efc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:53.923435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:53.976499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 734/1483] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:54:54.263009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aecd4e-fd68-4d53-9483-5fe4eb39b314 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:54.264544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4aecd4e-fd68-4d53-9483-5fe4eb39b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:54.288423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:54.367081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aecd4e-fd68-4d53-9483-5fe4eb39b314 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:54.439744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:54.440347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:54.440625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:54.440896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:54.442513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:54.517495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:54.551643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:54.553095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb56b4b-6e98-4231-ac85-4c740e166d89 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:54.686833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 750/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:56.905675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8557-e4b7-43f6-9756-8a013f423cb2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:56.906268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8557-e4b7-43f6-9756-8a013f423cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:56.906268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8557-e4b7-43f6-9756-8a013f423cb2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:56.907578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 735/1485] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:56.912861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681acb8c-6ea0-447f-9e08-5c934a32214e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:56.913283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681acb8c-6ea0-447f-9e08-5c934a32214e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:56.913521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681acb8c-6ea0-447f-9e08-5c934a32214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:56.980706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-681acb8c-6ea0-447f-9e08-5c934a32214e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:56.981929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-681acb8c-6ea0-447f-9e08-5c934a32214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:57.074950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 751/1486] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:54:57.362641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8557-e4b7-43f6-9756-8a013f423cb2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:54:57.364148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d3af8557-e4b7-43f6-9756-8a013f423cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:57.465230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8557-e4b7-43f6-9756-8a013f423cb2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:54:57.535244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:54:57.536711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:54:57.537360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:57.537640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:57.537931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:57.540095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:54:57.646917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:54:57.648262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3030-3413-4a5f-b0e2-3e0a56f3daa5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:54:57.665703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 736/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:54:59.921490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855495b-a891-4aa8-b539-859ca6ea4e0c None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:59.922076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855495b-a891-4aa8-b539-859ca6ea4e0c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:59.922076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855495b-a891-4aa8-b539-859ca6ea4e0c None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:59.923412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 752/1488] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:54:59.928865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406a26a-a167-46ce-9c10-7b2983a8db2b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:54:59.929284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406a26a-a167-46ce-9c10-7b2983a8db2b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:54:59.929429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406a26a-a167-46ce-9c10-7b2983a8db2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:54:59.994831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a406a26a-a167-46ce-9c10-7b2983a8db2b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:54:59.996668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a406a26a-a167-46ce-9c10-7b2983a8db2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:55:00.048969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:55:00.092593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 737/1489] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:55:00.391453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855495b-a891-4aa8-b539-859ca6ea4e0c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:55:00.392986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e855495b-a891-4aa8-b539-859ca6ea4e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:55:00.429979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:55:00.495974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855495b-a891-4aa8-b539-859ca6ea4e0c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:55:00.568014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:55:00.568621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:55:00.568907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:55:00.569178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:55:00.570802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:55:00.649579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:55:00.678914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:55:00.680348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f41a6-55c5-43b3-adaa-e1eeff524c4d service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:55:00.697521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 753/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:55:03.108878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81729a-10df-4f0b-8e1f-b5933271213a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:55:03.109434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81729a-10df-4f0b-8e1f-b5933271213a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:55:03.109434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81729a-10df-4f0b-8e1f-b5933271213a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:55:03.110832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 738/1491] 10.222.0.61 () {54 vars in 927 bytes} [Sat Sep 24 14:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:55:03.116442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dc952-d201-479c-b319-38b2d98cfaa8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:55:03.116864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dc952-d201-479c-b319-38b2d98cfaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:55:03.116990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dc952-d201-479c-b319-38b2d98cfaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:55:03.181210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b26dc952-d201-479c-b319-38b2d98cfaa8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:55:03.182477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b26dc952-d201-479c-b319-38b2d98cfaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:55:03.276540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 754/1492] 10.222.0.61 () {58 vars in 1024 bytes} [Sat Sep 24 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:55:03.569141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81729a-10df-4f0b-8e1f-b5933271213a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:55:03.570719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ed81729a-10df-4f0b-8e1f-b5933271213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:55:03.675056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81729a-10df-4f0b-8e1f-b5933271213a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:55:03.753456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:55:03.754091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:55:03.754368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:55:03.754637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:55:03.756226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:55:03.766016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:55:03.863785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:55:03.865393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66509ca6-f009-4c15-b746-a47f72af1d19 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:55:03.882874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 739/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:55:13.275029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=82 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:55:13.880609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=50 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:24.163861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:24.164450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:24.164450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:24.238818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:56:24.240025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:24.334910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 755/1494] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:56:24.342860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808613-07bb-4604-a945-26915027d229 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:24.343390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808613-07bb-4604-a945-26915027d229 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:24.343390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08808613-07bb-4604-a945-26915027d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:24.418335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-08808613-07bb-4604-a945-26915027d229 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:56:24.419534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-08808613-07bb-4604-a945-26915027d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:24.514594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 740/1495] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:56:24.519253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:24.520438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:24.531529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06192b-2a68-4696-840c-8014dbdef199 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:24.593411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:24.627398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:24.627992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:24.628264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:24.628528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:24.629154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:56:24.629450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:24.630478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:24.630478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:56:24.630478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:56:24.631349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:24.643046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 756/1496] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:56:24.651353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08808613-07bb-4604-a945-26915027d229 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:24.661876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08808613-07bb-4604-a945-26915027d229 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:24.722520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:24.756150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:24.756738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:24.757012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:24.757280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:24.757963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:24.759701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:24.759701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 14:56:24.759701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 14:56:24.760884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:24.816114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 741/1497] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:56:24 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:56:24.820192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:24.821686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:24.854703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7d2ca-cede-41e9-80d9-901f8153db83 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:24.957088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:24.957692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:24.957995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:24.958282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:24.958910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:24.959935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:24.959935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:56:24.959935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:56:24.961157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:25.003990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 757/1498] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:56:24 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:56:25.008138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:25.009699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:25.071221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf002-1141-48bc-a332-fafbc1e655d8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:25.153202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:25.164033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:25.164633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.164909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.165269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.166468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:25.167467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:25.167467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 14:56:25.167467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 14:56:25.168350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:25.180498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 742/1499] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:56:25.184574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:25.186096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:25.197205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da575b56-a836-4586-b911-b9312c12b923 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:25.221688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:25.290751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:25.291367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.291645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.291913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.292547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:25.293551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:25.293551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:56:25.293551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:56:25.294487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8350e-afbc-4a47-81e9-7a68af1cdbb3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:25.306503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 758/1500] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:56:25.310570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:25.312079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:25.345557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aa39e-4f01-4fd4-acd7-b860008a03b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:25.441069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:25.441728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.442049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.442330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] PATH_INFO: `/v3/projects/aa368dfc64524e08b4ee4d38661660ac/users/015a3df82b4946c393b0f72315a1cc09/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.465590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa368dfc64524e08b4ee4d38661660ac, user_id=015a3df82b4946c393b0f72315a1cc09, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:25.466654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:25.466654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 14:56:25.466654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 14:56:25.468598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e099-ca77-4126-93ce-40edd478b54d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:25.493355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 743/1501] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:56:25 2022] PUT /identity/v3/projects/aa368dfc64524e08b4ee4d38661660ac/users/015a3df82b4946c393b0f72315a1cc09/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:56:25.505365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb7328-658c-4954-b7e2-3664a56fe92a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.505652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb7328-658c-4954-b7e2-3664a56fe92a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.505956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb7328-658c-4954-b7e2-3664a56fe92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.561320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6fcb7328-658c-4954-b7e2-3664a56fe92a None None] MFA Rules not processed for user `015a3df82b4946c393b0f72315a1cc09`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:56:25.562534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6fcb7328-658c-4954-b7e2-3664a56fe92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:25.637114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 759/1502] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:56:25.650573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6bc14-e34a-4cfc-a7e6-35e8f9bb0b43 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.650860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6bc14-e34a-4cfc-a7e6-35e8f9bb0b43 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.651134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6bc14-e34a-4cfc-a7e6-35e8f9bb0b43 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.652461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 744/1503] 10.222.0.61 () {54 vars in 963 bytes} [Sat Sep 24 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:56:25.657508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84326e-1bbd-4fbf-8187-c8fa9d5f92ec None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.657862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84326e-1bbd-4fbf-8187-c8fa9d5f92ec None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.658195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84326e-1bbd-4fbf-8187-c8fa9d5f92ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.688853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:25.733764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dd84326e-1bbd-4fbf-8187-c8fa9d5f92ec None None] MFA Rules not processed for user `83758a542c2e4314871bf03fd2cca8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:56:25.734989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dd84326e-1bbd-4fbf-8187-c8fa9d5f92ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:25.808198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 760/1504] 10.222.0.61 () {58 vars in 1060 bytes} [Sat Sep 24 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:56:25.815882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6bc14-e34a-4cfc-a7e6-35e8f9bb0b43 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:25.817350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c0c6bc14-e34a-4cfc-a7e6-35e8f9bb0b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:25.868229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:25.922127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6bc14-e34a-4cfc-a7e6-35e8f9bb0b43 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:25.995450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:25.996053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:25.996329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:25.996601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:25.998271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:26.096261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:26.097689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:26.101212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:26.116385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 745/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:56:26.161621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182188c-8179-40fd-acaa-eda2fcc01afb None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:26.161952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182188c-8179-40fd-acaa-eda2fcc01afb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:26.162211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182188c-8179-40fd-acaa-eda2fcc01afb None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:26.163445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 761/1506] 10.222.0.61 () {52 vars in 830 bytes} [Sat Sep 24 14:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:56:26.167656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:26.168858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:26.265745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228000e-5239-47e2-9876-d6c700439670 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:26.357022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:26.357684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:26.358003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:26.358278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] PATH_INFO: `/v3/users/015a3df82b4946c393b0f72315a1cc09` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:26.371736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=015a3df82b4946c393b0f72315a1cc09)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:26.372761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:26.372761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 14:56:26.372761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 14:56:26.374004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:26.383706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:26.440696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015a3df82b4946c393b0f72315a1cc09'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:26.449857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015a3df82b4946c393b0f72315a1cc09'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:26.473456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015a3df82b4946c393b0f72315a1cc09'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:26.473922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015a3df82b4946c393b0f72315a1cc09'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:26.488713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015a3df82b4946c393b0f72315a1cc09'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:26.494756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 746/1507] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 14:56:26 2022] DELETE /identity/v3/users/015a3df82b4946c393b0f72315a1cc09 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:56:26.499195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:26.499691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:26.499800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:26.556220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:56:26.557382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:26.632366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:26.645032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 762/1508] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:56:26.655854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:26.657262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:26.758808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c66aa-58d5-4d0a-bad1-90e88aa845b0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:26.812607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:26.832292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:26.832902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:26.833180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:26.833450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:26.835050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:26.943212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:26.944639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b668c-3890-408a-9eb2-ed84a5e00c7b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:26.965350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 747/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:56:27.321514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:56:27.323326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:56:27.334509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92d98c-4753-48f2-9c26-32861d0eae38 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:56:27.427422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:56:27.428039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:56:27.428314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:56:27.428578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] PATH_INFO: `/v3/projects/aa368dfc64524e08b4ee4d38661660ac` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:56:27.438679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa368dfc64524e08b4ee4d38661660ac)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:56:27.439692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:56:27.439692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:56:27.439692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:56:27.440883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:56:27.496656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa368dfc64524e08b4ee4d38661660ac', 'request_id': 'req-ac89a760-387b-414e-8652-223a108de949'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:27.502813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa368dfc64524e08b4ee4d38661660ac', 'request_id': 'req-ac89a760-387b-414e-8652-223a108de949'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:27.517260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa368dfc64524e08b4ee4d38661660ac', 'request_id': 'req-ac89a760-387b-414e-8652-223a108de949'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:27.542859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] The token cache is being invalidate because project aa368dfc64524e08b4ee4d38661660ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:56:27.543147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:27.543603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ac89a760-387b-414e-8652-223a108de949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:56:27.544963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 763/1510] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 14:56:27 2022] DELETE /identity/v3/projects/aa368dfc64524e08b4ee4d38661660ac => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:56:36.962479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=62 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:56:37.540773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:57:55.166831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f93af4-afad-4188-9b80-0eabbab05fd5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:55.167419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f93af4-afad-4188-9b80-0eabbab05fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:55.167419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f93af4-afad-4188-9b80-0eabbab05fd5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:55.168735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:57:55.174156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96588d5-71fd-4ae3-8b9b-e9a3d42a598a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:55.174591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96588d5-71fd-4ae3-8b9b-e9a3d42a598a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:55.174741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96588d5-71fd-4ae3-8b9b-e9a3d42a598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:55.253773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f96588d5-71fd-4ae3-8b9b-e9a3d42a598a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:57:55.254988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f96588d5-71fd-4ae3-8b9b-e9a3d42a598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:57:55.356605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:57:55.511197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f93af4-afad-4188-9b80-0eabbab05fd5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:57:55.513398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c4f93af4-afad-4188-9b80-0eabbab05fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:57:55.632304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f93af4-afad-4188-9b80-0eabbab05fd5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:57:55.637953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6aab4-64b5-456f-8946-a6c100dc6eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:57:55.638560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6aab4-64b5-456f-8946-a6c100dc6eeb service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:55.638842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6aab4-64b5-456f-8946-a6c100dc6eeb service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:55.639114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6aab4-64b5-456f-8946-a6c100dc6eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:55.647602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6aab4-64b5-456f-8946-a6c100dc6eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:57:55.649158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6aab4-64b5-456f-8946-a6c100dc6eeb service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:57:55.658805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 749/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:57:56.719623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead967e-e25a-477d-9221-45fe540a4f45 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:56.720681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead967e-e25a-477d-9221-45fe540a4f45 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:56.720681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead967e-e25a-477d-9221-45fe540a4f45 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:56.722049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:57:56.728188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadd5bd-c9fc-48dc-b14f-0aae0aa9c437 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:56.728621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadd5bd-c9fc-48dc-b14f-0aae0aa9c437 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:56.728735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadd5bd-c9fc-48dc-b14f-0aae0aa9c437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:56.748600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4aadd5bd-c9fc-48dc-b14f-0aae0aa9c437 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:57:56.749779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4aadd5bd-c9fc-48dc-b14f-0aae0aa9c437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:57:56.757536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:57:56.900232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09598ed-c04f-451f-989e-366badfbdc02 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:56.900524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09598ed-c04f-451f-989e-366badfbdc02 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:56.900798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09598ed-c04f-451f-989e-366badfbdc02 None None] PATH_INFO: `/v3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:56.902160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 766/1516] 10.222.0.72 () {54 vars in 925 bytes} [Sat Sep 24 14:57:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:57:56.909326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b736a-8d18-4a65-a50b-1f5f29d1c14d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:56.909615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b736a-8d18-4a65-a50b-1f5f29d1c14d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:56.909892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b736a-8d18-4a65-a50b-1f5f29d1c14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:56.913209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09598ed-c04f-451f-989e-366badfbdc02 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:57:56.929346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09598ed-c04f-451f-989e-366badfbdc02 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:57:56.936670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b860a4-5a07-43d6-b25d-51560033b8a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:57:56.937282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b860a4-5a07-43d6-b25d-51560033b8a5 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:57:56.937564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b860a4-5a07-43d6-b25d-51560033b8a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:57:56.937836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b860a4-5a07-43d6-b25d-51560033b8a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:57:56.946380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b860a4-5a07-43d6-b25d-51560033b8a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:57:56.947983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b860a4-5a07-43d6-b25d-51560033b8a5 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:57:56.957322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 767/1517] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:57:56.973302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6a3b736a-8d18-4a65-a50b-1f5f29d1c14d None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:57:56.974767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a3b736a-8d18-4a65-a50b-1f5f29d1c14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:57:57.034229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:57:57.054198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 751/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Sat Sep 24 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:58:06.010839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8dd93-50c1-459d-919d-ac57ac2a1a52 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:06.011272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8dd93-50c1-459d-919d-ac57ac2a1a52 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:06.011390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8dd93-50c1-459d-919d-ac57ac2a1a52 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:06.012702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 768/1519] 10.222.0.61 () {54 vars in 961 bytes} [Sat Sep 24 14:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:06.018261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f2bca-f4a5-4898-a802-a740b0ed36fc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:06.018728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f2bca-f4a5-4898-a802-a740b0ed36fc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:06.018834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f2bca-f4a5-4898-a802-a740b0ed36fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:06.038847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-af2f2bca-f4a5-4898-a802-a740b0ed36fc None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:06.040076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af2f2bca-f4a5-4898-a802-a740b0ed36fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:06.048640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 752/1520] 10.222.0.61 () {58 vars in 1058 bytes} [Sat Sep 24 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:06.056065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8dd93-50c1-459d-919d-ac57ac2a1a52 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:06.069148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8dd93-50c1-459d-919d-ac57ac2a1a52 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:06.074024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b68a3-4610-4610-94e2-3fbe4ddf7b1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:06.074604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b68a3-4610-4610-94e2-3fbe4ddf7b1a service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:06.074883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b68a3-4610-4610-94e2-3fbe4ddf7b1a service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:06.075157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b68a3-4610-4610-94e2-3fbe4ddf7b1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:06.083625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b68a3-4610-4610-94e2-3fbe4ddf7b1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:06.085183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b68a3-4610-4610-94e2-3fbe4ddf7b1a service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:06.094558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 769/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:08.418016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8b9cb-6ab3-4f5f-976c-3e5e58f34280 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:08.418602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8b9cb-6ab3-4f5f-976c-3e5e58f34280 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:08.418602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8b9cb-6ab3-4f5f-976c-3e5e58f34280 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:08.419928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:08.426741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995ba45-dcbb-4e70-b5a0-c2f32c0ccedc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:08.427185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995ba45-dcbb-4e70-b5a0-c2f32c0ccedc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:08.427289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995ba45-dcbb-4e70-b5a0-c2f32c0ccedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:08.447342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7995ba45-dcbb-4e70-b5a0-c2f32c0ccedc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:08.448532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7995ba45-dcbb-4e70-b5a0-c2f32c0ccedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:08.456315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:08.615020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8b9cb-6ab3-4f5f-976c-3e5e58f34280 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:08.629657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8b9cb-6ab3-4f5f-976c-3e5e58f34280 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:08.635151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a58d5a-de66-4a47-9356-b1541c83ca0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:08.635755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a58d5a-de66-4a47-9356-b1541c83ca0f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:08.636041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a58d5a-de66-4a47-9356-b1541c83ca0f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:08.636343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a58d5a-de66-4a47-9356-b1541c83ca0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:08.644381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a58d5a-de66-4a47-9356-b1541c83ca0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:08.645987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a58d5a-de66-4a47-9356-b1541c83ca0f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:08.655179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 754/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:09.730970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d5adf-5090-4738-ad8d-f8b034f120d1 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:09.731572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d5adf-5090-4738-ad8d-f8b034f120d1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:09.731572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d5adf-5090-4738-ad8d-f8b034f120d1 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:09.732850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:09.739808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2e19-cf19-46f4-ac71-0d415bd8cb65 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:09.740253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2e19-cf19-46f4-ac71-0d415bd8cb65 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:09.740356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2e19-cf19-46f4-ac71-0d415bd8cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:09.760008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7a9d2e19-cf19-46f4-ac71-0d415bd8cb65 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:09.761209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a9d2e19-cf19-46f4-ac71-0d415bd8cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:09.768943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:09.924151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d5adf-5090-4738-ad8d-f8b034f120d1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:09.938299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d5adf-5090-4738-ad8d-f8b034f120d1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:09.943560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eb67f-26a7-4394-949e-15113f5f9aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:09.944143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eb67f-26a7-4394-949e-15113f5f9aca service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:09.944418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eb67f-26a7-4394-949e-15113f5f9aca service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:09.944686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eb67f-26a7-4394-949e-15113f5f9aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:09.952435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eb67f-26a7-4394-949e-15113f5f9aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:09.954027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eb67f-26a7-4394-949e-15113f5f9aca service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:09.963498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 772/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:13.941502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4e18-ab92-43da-88e6-359b3aeaf03d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:13.942010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4e18-ab92-43da-88e6-359b3aeaf03d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:13.942122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4e18-ab92-43da-88e6-359b3aeaf03d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:13.943420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:13.948632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2becc-c225-4a98-a370-5c861eba66e7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:13.949055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2becc-c225-4a98-a370-5c861eba66e7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:13.949193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2becc-c225-4a98-a370-5c861eba66e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:13.968478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-63a2becc-c225-4a98-a370-5c861eba66e7 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:13.969671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-63a2becc-c225-4a98-a370-5c861eba66e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:13.978262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:14.260082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4e18-ab92-43da-88e6-359b3aeaf03d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:14.261745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b7dd4e18-ab92-43da-88e6-359b3aeaf03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:14.339322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4e18-ab92-43da-88e6-359b3aeaf03d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:14.342859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ab9f7-802d-4fad-95b8-d53c4327963d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:14.343454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ab9f7-802d-4fad-95b8-d53c4327963d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:14.343737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ab9f7-802d-4fad-95b8-d53c4327963d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:14.344011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ab9f7-802d-4fad-95b8-d53c4327963d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:14.353289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ab9f7-802d-4fad-95b8-d53c4327963d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:14.354750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ab9f7-802d-4fad-95b8-d53c4327963d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:14.364758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 757/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:16.510008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fe7f4-b98b-462b-a13e-9bf9e5525e4c None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:16.510541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fe7f4-b98b-462b-a13e-9bf9e5525e4c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:16.510541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fe7f4-b98b-462b-a13e-9bf9e5525e4c None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:16.511903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:16.517023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ff79a-f83a-44ea-b649-fde732d0de7c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:16.517465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ff79a-f83a-44ea-b649-fde732d0de7c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:16.517596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ff79a-f83a-44ea-b649-fde732d0de7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:16.539372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a74ff79a-f83a-44ea-b649-fde732d0de7c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:16.540580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a74ff79a-f83a-44ea-b649-fde732d0de7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:16.548383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:16.835158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fe7f4-b98b-462b-a13e-9bf9e5525e4c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:16.846769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fe7f4-b98b-462b-a13e-9bf9e5525e4c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:16.850337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51f6f4-24ee-435f-bc61-0ac78b95bc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:16.850958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51f6f4-24ee-435f-bc61-0ac78b95bc12 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:16.851239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51f6f4-24ee-435f-bc61-0ac78b95bc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:16.851512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51f6f4-24ee-435f-bc61-0ac78b95bc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:16.860178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51f6f4-24ee-435f-bc61-0ac78b95bc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:16.861623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51f6f4-24ee-435f-bc61-0ac78b95bc12 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:16.871383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 775/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:20.514748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a80edd-1a6b-438a-a018-637e1b2c64ea None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:20.515314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a80edd-1a6b-438a-a018-637e1b2c64ea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:20.515314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a80edd-1a6b-438a-a018-637e1b2c64ea None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:20.516692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:20.523665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c9a9c-9ee8-4eae-a9b8-8d6e0449ca8d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:20.524115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c9a9c-9ee8-4eae-a9b8-8d6e0449ca8d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:20.524221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c9a9c-9ee8-4eae-a9b8-8d6e0449ca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:20.532156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:58:20.546161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fb8c9a9c-9ee8-4eae-a9b8-8d6e0449ca8d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:20.547368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fb8c9a9c-9ee8-4eae-a9b8-8d6e0449ca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:20.555207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:20.859607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a80edd-1a6b-438a-a018-637e1b2c64ea None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:20.861325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62a80edd-1a6b-438a-a018-637e1b2c64ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:20.938695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a80edd-1a6b-438a-a018-637e1b2c64ea None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:20.942765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d32e6-f781-47e0-9493-db22488e3ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:20.943364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d32e6-f781-47e0-9493-db22488e3ccc service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:20.943645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d32e6-f781-47e0-9493-db22488e3ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:20.943919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d32e6-f781-47e0-9493-db22488e3ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:20.952519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d32e6-f781-47e0-9493-db22488e3ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:20.954004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d32e6-f781-47e0-9493-db22488e3ccc service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:20.963885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 760/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:23.114699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb0a90-a63a-4914-baec-4fa6b2c3e022 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:23.115229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb0a90-a63a-4914-baec-4fa6b2c3e022 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:23.115229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb0a90-a63a-4914-baec-4fa6b2c3e022 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:23.116585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:58:23.121875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6281a1-b528-477e-97af-6da793eed1c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:23.122290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6281a1-b528-477e-97af-6da793eed1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:23.122442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6281a1-b528-477e-97af-6da793eed1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:23.142722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9f6281a1-b528-477e-97af-6da793eed1c5 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:58:23.143920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f6281a1-b528-477e-97af-6da793eed1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:23.151716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:58:23.453447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb0a90-a63a-4914-baec-4fa6b2c3e022 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:58:23.455664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0ecb0a90-a63a-4914-baec-4fa6b2c3e022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:58:23.471491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb0a90-a63a-4914-baec-4fa6b2c3e022 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:58:23.475007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a63b0-7187-4319-ab9c-8f22c236268d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:58:23.475599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a63b0-7187-4319-ab9c-8f22c236268d service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:58:23.475883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a63b0-7187-4319-ab9c-8f22c236268d service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:58:23.476158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a63b0-7187-4319-ab9c-8f22c236268d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:58:23.484844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a63b0-7187-4319-ab9c-8f22c236268d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:58:23.486314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a63b0-7187-4319-ab9c-8f22c236268d service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:58:23.495794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 778/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:58:33.137705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:58:33.491676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=8 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:48.392042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.392835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.392835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.417679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.419011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.428411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 762/1540] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:48.437426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.438040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.438040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.476985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.478281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.498242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 779/1541] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:48.503395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:48.518587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ba603-8993-4460-96b9-34b0cc3f6161 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:48.534761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:48.535397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.535681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.535955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.536978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:48.537287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:48.538527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:59:48.538527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 14:59:48.538527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 14:59:48.539458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:48.556031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 763/1542] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:48.564547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:48.578621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb54b-221b-435d-b91c-dee280e323b7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:48.586802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:48.587453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.587732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.588009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.588849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:48.589931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:59:48.589931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:59:48.589931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:59:48.591149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c944200-de7d-41f5-884d-92ebfa0b6551 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:48.655705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 780/1543] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:48.660841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:48.676662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f72b7-a9ca-492b-aee6-32d7227af380 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:48.766668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:48.767562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.768404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.768681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.769321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:48.770363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:59:48.770363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 14:59:48.770363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 14:59:48.771578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaa51c-5cd1-4623-a494-99e6c8b4ae73 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:48.775322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6652-e037-4185-8543-1259571d9106 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.775614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6652-e037-4185-8543-1259571d9106 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.775896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6652-e037-4185-8543-1259571d9106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.812876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 764/1544] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:48.814281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9f997-fe82-4ce5-ab00-de5623eadd49 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.814609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9f997-fe82-4ce5-ab00-de5623eadd49 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.814890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9f997-fe82-4ce5-ab00-de5623eadd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.832271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4cec6652-e037-4185-8543-1259571d9106 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.833497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4cec6652-e037-4185-8543-1259571d9106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.853176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 781/1545] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:59:48.857977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6652-e037-4185-8543-1259571d9106 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:48.861161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-47b9f997-fe82-4ce5-ab00-de5623eadd49 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.865931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-47b9f997-fe82-4ce5-ab00-de5623eadd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.878121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6652-e037-4185-8543-1259571d9106 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:48.887126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 765/1546] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:48.888506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe25aa6-bbdb-4c6d-9e68-63cbda9fbb5b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.888797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe25aa6-bbdb-4c6d-9e68-63cbda9fbb5b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.889075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe25aa6-bbdb-4c6d-9e68-63cbda9fbb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.892321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253051-0f49-4227-abef-be930b58ae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:48.893622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253051-0f49-4227-abef-be930b58ae7c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.893945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253051-0f49-4227-abef-be930b58ae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.894208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253051-0f49-4227-abef-be930b58ae7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.895257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253051-0f49-4227-abef-be930b58ae7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:48.897572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253051-0f49-4227-abef-be930b58ae7c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:48.911121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 782/1547] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:48.912321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce2d08-efa4-42e0-8a1e-b5e6418f98cc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.912608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce2d08-efa4-42e0-8a1e-b5e6418f98cc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.912883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce2d08-efa4-42e0-8a1e-b5e6418f98cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.934880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8fe25aa6-bbdb-4c6d-9e68-63cbda9fbb5b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.936460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8fe25aa6-bbdb-4c6d-9e68-63cbda9fbb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.945426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 766/1548] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:48.946968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.947490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.947989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.950946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fbce2d08-efa4-42e0-8a1e-b5e6418f98cc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.952172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fbce2d08-efa4-42e0-8a1e-b5e6418f98cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.969561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 783/1549] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:48.971773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4fe37-b783-498b-beff-349fc7a09bc3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.972014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.972175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4fe37-b783-498b-beff-349fc7a09bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.972370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4fe37-b783-498b-beff-349fc7a09bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:48.973251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.983097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 767/1550] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:48.983968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:48.992962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0dd4fe37-b783-498b-beff-349fc7a09bc3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:48.993686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ec94-ddb9-41e4-bd32-b356273810cd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:48.994169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0dd4fe37-b783-498b-beff-349fc7a09bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:48.998485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3b91-7644-469a-ba0e-cd18bf501b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:48.999085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3b91-7644-469a-ba0e-cd18bf501b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:48.999397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3b91-7644-469a-ba0e-cd18bf501b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:48.999674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3b91-7644-469a-ba0e-cd18bf501b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.000321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af3b91-7644-469a-ba0e-cd18bf501b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.001937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af3b91-7644-469a-ba0e-cd18bf501b9a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.002967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 784/1551] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.004522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be1e5-c1bc-42f7-918f-0e5857ad2784 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.004807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be1e5-c1bc-42f7-918f-0e5857ad2784 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.005078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be1e5-c1bc-42f7-918f-0e5857ad2784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.015913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 768/1552] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:49.015913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.015913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.015913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.025434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0f1be1e5-c1bc-42f7-918f-0e5857ad2784 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.026709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0f1be1e5-c1bc-42f7-918f-0e5857ad2784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.033929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.034639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 785/1553] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.035089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.035800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53174d29-4968-4743-ad74-531aa9ea274e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.036081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53174d29-4968-4743-ad74-531aa9ea274e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.036382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53174d29-4968-4743-ad74-531aa9ea274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.043272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 769/1554] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.044060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.053293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ea7be-00c9-40da-ac06-49d66a4ea068 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.055775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-53174d29-4968-4743-ad74-531aa9ea274e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.056947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-53174d29-4968-4743-ad74-531aa9ea274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.057795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.058447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.058736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.059014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.059650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.059957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.061546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6412c8-0c32-4fac-8367-8ae09288fc07 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.065403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 786/1555] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.071689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.072141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.072505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.074255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 770/1556] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.075535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.075817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.076099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.092908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.094126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.095269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.096461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.101802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 787/1557] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.102604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.111487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 771/1558] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.112072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d030a-e611-4615-adf7-e83a91e393be None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.112266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.116946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.117541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.117825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.118127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.118774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.119083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.120672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec108c-588c-476a-9be4-1e08b6be19ce admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.122208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e345ac-b350-4a22-bc86-9bf0295e4e39 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.126773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.127983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.128289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.128563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] PATH_INFO: `/v3/projects/9b2919ee12fe488ea94db84e9af057a2/users/9f74f51060554db18bc96199222e9261/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.132844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 788/1559] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.134177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67675d-311d-40e5-a032-62168f92f245 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.134467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67675d-311d-40e5-a032-62168f92f245 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.134743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67675d-311d-40e5-a032-62168f92f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.145515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b2919ee12fe488ea94db84e9af057a2, user_id=9f74f51060554db18bc96199222e9261, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.148213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.154237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7c67675d-311d-40e5-a032-62168f92f245 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.155735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7c67675d-311d-40e5-a032-62168f92f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.162381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 772/1560] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:49 2022] PUT /identity/v3/projects/9b2919ee12fe488ea94db84e9af057a2/users/9f74f51060554db18bc96199222e9261/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:49.163091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.211490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:49.218118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 789/1561] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.219354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a752-21fc-4c9a-ac6b-38a2f8ae4bb4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.219642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a752-21fc-4c9a-ac6b-38a2f8ae4bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.219917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a752-21fc-4c9a-ac6b-38a2f8ae4bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.226825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e264d-55b2-4d9d-9793-9a58d19f91ac admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.231448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.232050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.232333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.232610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.233252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.233557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.235180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.238846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-aa13a752-21fc-4c9a-ac6b-38a2f8ae4bb4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.240028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aa13a752-21fc-4c9a-ac6b-38a2f8ae4bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.246753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 773/1562] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.247698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 790/1563] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.252821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.253071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.253355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.253648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.262730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aaa61a-ddd9-4d56-9ca5-a26a0ba9c921 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.267343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.268489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.268771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.269047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.269677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.271619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.273366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.274572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.282502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 791/1564] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.283308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.293613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f2d59-1766-42b2-9912-3916d1f92fad None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.298845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.299661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.300010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.300377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.301132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.301731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.304425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2342c-026d-4294-b469-8275af5da8c5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.306148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 774/1565] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.307091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.327232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 792/1566] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.327232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.327232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.327232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.340489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:49.341672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.369558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:49.374011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367de658-3d30-4f78-ab76-726c2bebd7ce admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.379382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.379980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.380261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.380534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.381163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.383259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.402850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 793/1567] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:49.403713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.413109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b103b-2d04-452c-a6d0-8b8e51602bc2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.417601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.418346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.418633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.421874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 775/1568] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.422371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.422371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.422550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.422647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.422647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.433768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 794/1569] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.434655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.446044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84fb97-9935-4baf-b2f7-c5e9d7893b40 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.483340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ef4ad-90d1-49d7-ac4b-6d82cb96b330 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.488898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.489576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.489869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.490204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.490849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.491161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.492772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.501920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.503082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.503357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.503624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.504238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.506120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 776/1570] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.506262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.510941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.521950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebacce1-c8c3-4fe0-8f2f-e5ddabbe7fef admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.527076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.527749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.528136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.528486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.529204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:59:49.529516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.531159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.537021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 795/1571] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.537021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.544222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 777/1572] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:59:49.545177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.556433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5ec8-1900-41b6-9a1d-42e930128321 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.579686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:49.596974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35589a2-f663-4776-86f9-d584b5bb0be4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.601605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.602235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.602519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.602794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.603424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.605352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.612565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.613443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.613720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.614037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.614670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.616535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.629642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 796/1573] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.629642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.629642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:49.650058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 778/1574] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 361 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.650058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.672249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:49.691757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094093b-c92e-4748-8b55-04b75120035a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.708683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ba1a1-1f28-4a07-9c8e-fe79970db51c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.713371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.714009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.714293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.714569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.715198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.717111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.741618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 779/1575] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.742554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.742707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.743262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.743496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.743774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.752922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7a1c8-0a2b-4ae1-89b2-b4b94f49116f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.754171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.755626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.757801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.758495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.758846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.759198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.760040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.762023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.765001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 797/1576] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:49.765933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.775931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893da5b-84ba-41d8-987f-18b0cd86d11f service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.781611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.781797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.782073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.782346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.782992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.785472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.794867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 780/1577] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.794867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.816447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 798/1578] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.817349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.867191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d85f1-0d75-45c2-a798-1d3cae41c45f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.887955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d8b02-51f8-4c1e-ad62-37a8fcca75ad admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:49.893655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.894527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.894989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.895547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.895819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:49.896384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.898430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.932728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 799/1579] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.932728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:49.948230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:49.948907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:49.949128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:49.949421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:49.950095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:49.953986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:49.989116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 781/1580] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:49.990031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.000469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafba77-590c-4398-8e1d-8ad6fd459518 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.010030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:50.014276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59ee18-e21c-4a82-9054-d7dfee8c58dd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.020426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.021218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.021555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.021897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.022812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.024697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.036434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 800/1581] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.037283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.060027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af152156-7d97-4118-a573-ec53c3cdb7de admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.064682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.065989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.066157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.066434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.066726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.066855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.067126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.067390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.067515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.068295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.069319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.069929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.080840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 801/1582] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.083356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.094234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf9a45-04c0-4c64-9955-0537d597ca11 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.098634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 782/1583] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:49 2022] POST /identity/v3/users => generated 281 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:50.098634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.102596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.103512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.104218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.107590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.107590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.107590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.112669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ef057-3409-4dd6-891d-5f7bedecb66e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.118672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.119533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.119919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.120302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.121014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.123879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.144640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 802/1584] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:50 2022] POST /identity/v3/users => generated 289 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:50.145623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.156266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 783/1585] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:50 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:50.157440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.159010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5c613-c75a-4484-a56f-4b823d27096e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.167142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.167799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.168095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.168342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.168976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.170488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece55cdc-8fa1-440b-a6b0-446a1871ad4d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.171298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.175548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.176481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.176924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.177344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.178183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.180152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.197773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 784/1586] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.197773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.202048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 803/1587] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:50 2022] POST /identity/v3/users => generated 295 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:50.202918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.214220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b620b57-a812-4423-a3a9-c9e6a48908bb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.216449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8d2c5-3bda-46ff-b331-2ef206bc982e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.219936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.221592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.221887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.222262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.222514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.222794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.223062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.223327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.223701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.224280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.225304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.226209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.235853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 804/1588] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.238255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 785/1589] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.239437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.240124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.249963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f298f01-2573-415f-9e5c-214efbc7bb2f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.250423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c479f-66e2-468a-8301-5e468ed50c4f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.255368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.255974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.256258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.256534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.257174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.257505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.258386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.258845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.258974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.259549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.260423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.262429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.270634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 805/1590] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.271535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.275266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 786/1591] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.276261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.284743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8834-d1b2-496e-adb6-fca53d9b3a61 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.289746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.290412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.290664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eded8e-c28e-44d9-812f-a15ab2bf8c21 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.290775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.290953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.291597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.293197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.296742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.297562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.298062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.298545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.299220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.300794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.304659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 806/1592] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.305477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.315216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 787/1593] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.315522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e97705-cde6-46fc-ab91-12c4a3c916ae admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.319330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.321307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.321930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.322239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.322516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.323172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.324756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.329293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbce74-d3fa-4b99-ac96-1b399a2b6e7a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.334859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.335510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.335797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.336074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] PATH_INFO: `/v3/projects/c2e445f0acdd4718a39a8aef7043802b/users/cbc9c7ef2b4345e6bc3ec01935dc697b/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.340074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 807/1594] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.340074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.351695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fac22-c141-4358-8e01-874eb3885d65 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.354245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e445f0acdd4718a39a8aef7043802b, user_id=cbc9c7ef2b4345e6bc3ec01935dc697b, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.356941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.357220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.357705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.357930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.358261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] PATH_INFO: `/v3/projects/0fafaf0d34704d73b3cfd43bf4131dda/users/e2601f50494749caa10565f0320a257f/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.373448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 788/1595] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/c2e445f0acdd4718a39a8aef7043802b/users/cbc9c7ef2b4345e6bc3ec01935dc697b/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:50.374801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.388240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fafaf0d34704d73b3cfd43bf4131dda, user_id=e2601f50494749caa10565f0320a257f, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.389326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:59:50.389326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 14:59:50.389326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 14:59:50.391351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.407977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 808/1596] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/0fafaf0d34704d73b3cfd43bf4131dda/users/e2601f50494749caa10565f0320a257f/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:50.409647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.426145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:50.477438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f4f2f-c858-4ffd-ac23-4fda653ada62 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.478033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a857d1-c1b5-4dc1-abc9-dfc6e848f364 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.482618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.483671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.483671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.483856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.484684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.484801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.486900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.498401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.498851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.498979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 809/1597] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.499518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.499914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.500531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.513146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed73b-2b1a-4211-8ff5-8897eccaeffb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.519140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.519883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.520822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.520822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.521220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.522979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.523287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.537768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 810/1598] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.539131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.541544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 789/1599] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.543547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.555016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc25f89-49d5-4321-8757-ff58d6b7f29c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.561482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e99b6-2589-45d3-a7f1-c3b24698b4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.562059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba6f5-76c3-4ebc-8503-565ac9ca3277 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.562815 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e99b6-2589-45d3-a7f1-c3b24698b4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.562952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e99b6-2589-45d3-a7f1-c3b24698b4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.563377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e99b6-2589-45d3-a7f1-c3b24698b4ec admin admin] PATH_INFO: `/v3/projects/578268f34d874bf9835e8ec4cb31ebee/users/db5a0bddb68047d68f0c2dd1af0cd57b/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.570356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8932a5d-bdb2-4e6a-b2da-ddeb40a39678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.570984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8932a5d-bdb2-4e6a-b2da-ddeb40a39678 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.571833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8932a5d-bdb2-4e6a-b2da-ddeb40a39678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.572168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8932a5d-bdb2-4e6a-b2da-ddeb40a39678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.574317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8932a5d-bdb2-4e6a-b2da-ddeb40a39678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.577290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8932a5d-bdb2-4e6a-b2da-ddeb40a39678 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.586233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e99b6-2589-45d3-a7f1-c3b24698b4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=578268f34d874bf9835e8ec4cb31ebee, user_id=db5a0bddb68047d68f0c2dd1af0cd57b, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.589111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e99b6-2589-45d3-a7f1-c3b24698b4ec admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.591842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 790/1600] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:50.593408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.593704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.594014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.603661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:50.618042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 811/1601] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/578268f34d874bf9835e8ec4cb31ebee/users/db5a0bddb68047d68f0c2dd1af0cd57b/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:50.620053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.620346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.620624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.637623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] MFA Rules not processed for user `cbc9c7ef2b4345e6bc3ec01935dc697b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:50.638946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:50.656946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] MFA Rules not processed for user `e2601f50494749caa10565f0320a257f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:50.658235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:50.750590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 791/1602] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:50.750590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.750885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 812/1603] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:50.750885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.844333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781f931-f296-471b-9d7b-aee16fed1c53 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.849982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.850646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.850920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.851189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] PATH_INFO: `/v3/projects/727eb826972a49cf87453855388045f6/users/3f880497e66e4a94bb2b81ef5fdc4a72/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.858829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e1da5-eff0-46f8-a70b-5ad884a68591 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:50.865109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:50.865951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:50.866304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:50.866724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] PATH_INFO: `/v3/projects/5a28fd8d62a54976b961c31176fed822/users/5b63ee5b9ac44eb39617b291ac869db7/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:50.868903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=727eb826972a49cf87453855388045f6, user_id=3f880497e66e4a94bb2b81ef5fdc4a72, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.872213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.884935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a28fd8d62a54976b961c31176fed822, user_id=5b63ee5b9ac44eb39617b291ac869db7, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:50.888357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:50.893286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 813/1604] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/727eb826972a49cf87453855388045f6/users/3f880497e66e4a94bb2b81ef5fdc4a72/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:50.895872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.915142 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 792/1605] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/5a28fd8d62a54976b961c31176fed822/users/5b63ee5b9ac44eb39617b291ac869db7/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:50.920226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:50.947554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:50.998342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320c3d-6b92-45f8-93ce-8205fbe933f1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.003397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eabf3a-e89e-43dc-9106-33c4be4605ba admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.007283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddd1fa-4222-4a69-85b0-602862162eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.008254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd1fa-4222-4a69-85b0-602862162eab admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.008746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd1fa-4222-4a69-85b0-602862162eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.009232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd1fa-4222-4a69-85b0-602862162eab admin admin] PATH_INFO: `/v3/projects/ea560f6e50034738b03e588ada15e940/users/cdbc898ffd4f4d5d87c3684fcc17313d/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.009344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.009937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.010227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.010503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] PATH_INFO: `/v3/projects/da06115b1d3a4938824cc4c1b2460454/users/634fd179e81745758f35c9f6fd6c9b9e/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.029485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddd1fa-4222-4a69-85b0-602862162eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea560f6e50034738b03e588ada15e940, user_id=cdbc898ffd4f4d5d87c3684fcc17313d, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.030450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da06115b1d3a4938824cc4c1b2460454, user_id=634fd179e81745758f35c9f6fd6c9b9e, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.033347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.033632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddd1fa-4222-4a69-85b0-602862162eab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.036923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:51.053229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 793/1606] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/ea560f6e50034738b03e588ada15e940/users/cdbc898ffd4f4d5d87c3684fcc17313d/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:51.054654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 814/1607] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:50 2022] PUT /identity/v3/projects/da06115b1d3a4938824cc4c1b2460454/users/634fd179e81745758f35c9f6fd6c9b9e/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:51.055084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1e3ed-1138-4d2b-993a-4d3e00f3e08b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.055479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1e3ed-1138-4d2b-993a-4d3e00f3e08b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.055817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1e3ed-1138-4d2b-993a-4d3e00f3e08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.056381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.087964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-55c1e3ed-1138-4d2b-993a-4d3e00f3e08b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.089370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-55c1e3ed-1138-4d2b-993a-4d3e00f3e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.154331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0fc8e-158e-49a5-8d58-f0133a9cfadc admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.170569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a30b1-ecca-4086-8a55-20bcd63f9bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.171931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a30b1-ecca-4086-8a55-20bcd63f9bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.172110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a30b1-ecca-4086-8a55-20bcd63f9bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.172343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a30b1-ecca-4086-8a55-20bcd63f9bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.172994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a30b1-ecca-4086-8a55-20bcd63f9bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.175818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a30b1-ecca-4086-8a55-20bcd63f9bd0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.178539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 794/1608] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.183722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4aa2e-7773-4b2a-91ac-d8b98a9ebd1a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.184206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4aa2e-7773-4b2a-91ac-d8b98a9ebd1a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.184577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4aa2e-7773-4b2a-91ac-d8b98a9ebd1a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.187069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 795/1609] 10.222.0.61 () {54 vars in 961 bytes} [Sat Sep 24 14:59:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:59:51.190806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe36e53-eb26-4766-8434-7e1b3f8e0150 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.190806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe36e53-eb26-4766-8434-7e1b3f8e0150 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.190806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe36e53-eb26-4766-8434-7e1b3f8e0150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.221682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 815/1610] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:51.221682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6576915-b2c5-45ca-8034-52ebca095e4d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.222018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6576915-b2c5-45ca-8034-52ebca095e4d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.222127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6576915-b2c5-45ca-8034-52ebca095e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.224649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-afe36e53-eb26-4766-8434-7e1b3f8e0150 None None] MFA Rules not processed for user `3f880497e66e4a94bb2b81ef5fdc4a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.226227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-afe36e53-eb26-4766-8434-7e1b3f8e0150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.248209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b6576915-b2c5-45ca-8034-52ebca095e4d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.249734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b6576915-b2c5-45ca-8034-52ebca095e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.296924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 796/1611] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.298444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.298858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.299761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.322995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.324663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.350997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 816/1612] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.354021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b0fc6-0932-4e21-b4c8-a8beff9fbea7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.354362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b0fc6-0932-4e21-b4c8-a8beff9fbea7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.356833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b0fc6-0932-4e21-b4c8-a8beff9fbea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.377006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:51.392352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b62b0fc6-0932-4e21-b4c8-a8beff9fbea7 None None] MFA Rules not processed for user `634fd179e81745758f35c9f6fd6c9b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.393794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b62b0fc6-0932-4e21-b4c8-a8beff9fbea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.431733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 797/1613] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.433502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.509616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba6e7f-b541-448d-a335-9a61fc6d515c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.514811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.515447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.515755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.516032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.524870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 817/1614] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.524870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.524870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.524870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.536328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.537767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.549012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 798/1615] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:51.549856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.552810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:51.560386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f8380-1d91-4aff-a15c-233653ccd5bc service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.565291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554586cb-fcbd-4741-819e-d6a3af6badb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.565945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554586cb-fcbd-4741-819e-d6a3af6badb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.567098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554586cb-fcbd-4741-819e-d6a3af6badb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.567480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554586cb-fcbd-4741-819e-d6a3af6badb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.569157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554586cb-fcbd-4741-819e-d6a3af6badb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.571232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554586cb-fcbd-4741-819e-d6a3af6badb9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.591679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.593425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.606854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 799/1616] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/users => generated 295 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:59:51.612656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1baae5-4ad0-471f-9d6d-9fc4ffbf7f2a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.613194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1baae5-4ad0-471f-9d6d-9fc4ffbf7f2a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.613651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1baae5-4ad0-471f-9d6d-9fc4ffbf7f2a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.616302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 800/1617] 10.222.0.61 () {54 vars in 961 bytes} [Sat Sep 24 14:59:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:59:51.617115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1baae5-4ad0-471f-9d6d-9fc4ffbf7f2a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.633791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1baae5-4ad0-471f-9d6d-9fc4ffbf7f2a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.638832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.639634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.639954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.640239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.654600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.657727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.678410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 801/1618] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:51.680308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.698531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 818/1619] 10.222.0.61 () {58 vars in 1058 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.699816 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.704372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756d519-0797-482f-abf9-e725e877bc97 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.710564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.711570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.712070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.712665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.750060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.751259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.819472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 802/1620] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:51.819472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.819472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed312f-da16-45a3-85a1-7efafa299842 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.819472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4773a-f4c7-48b5-8973-4b80a68a76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.820149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4773a-f4c7-48b5-8973-4b80a68a76d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.820149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4773a-f4c7-48b5-8973-4b80a68a76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.820149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4773a-f4c7-48b5-8973-4b80a68a76d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.820149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4773a-f4c7-48b5-8973-4b80a68a76d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.820149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4773a-f4c7-48b5-8973-4b80a68a76d0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.820711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12718e-c2cc-40a3-b484-1312adae87ad None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.821881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.825748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.826088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.826333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.851500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 803/1621] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:51.853859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.853859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.857488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.882697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.886468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:51.887089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:51.888629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:51.901639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 804/1622] 10.222.0.61 () {58 vars in 1058 bytes} [Sat Sep 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:51.903088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.906854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 819/1623] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:51.907900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:51.920953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916625-72c9-4039-91f6-3070487141e8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.922418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba8fcf-70c7-46c7-b486-ad987415a0b4 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:51.942657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.942903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:51.944381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.945057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:51.945219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.945489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] PATH_INFO: `/v3/projects/2c6ad4f55a414db9a509be49690612b9/users/a7b4d6e47dca438084fa880f3f2f8293/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.946801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:51.947157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:51.978940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6ad4f55a414db9a509be49690612b9, user_id=a7b4d6e47dca438084fa880f3f2f8293, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:51.992388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:52.017070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 820/1624] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:51 2022] PUT /identity/v3/projects/2c6ad4f55a414db9a509be49690612b9/users/a7b4d6e47dca438084fa880f3f2f8293/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:59:52.023347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:52.073296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:52.075329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:52.089948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 805/1625] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:52.092358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:52.110982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea6c16-66b9-4bc6-bd29-f28730a43194 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:52.127834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:52.153354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398eca9-c3a2-4358-83c1-779af8b0d243 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:52.169215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262e67d-3e5a-4494-a691-96a350ab8053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:52.169840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262e67d-3e5a-4494-a691-96a350ab8053 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.170143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262e67d-3e5a-4494-a691-96a350ab8053 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.170429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262e67d-3e5a-4494-a691-96a350ab8053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.179165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:59:52.243918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:52.244549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.244835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.245117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.245837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:52.247494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:52.269111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262e67d-3e5a-4494-a691-96a350ab8053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:52.272377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262e67d-3e5a-4494-a691-96a350ab8053 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:52.278315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 806/1626] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 14:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:59:52.278315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:52.296385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 821/1627] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:52.300137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0d83-5ff2-43c1-b7be-100b75ea2f08 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:52.305660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:52.306413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.306699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.306983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] PATH_INFO: `/v3/projects/2c6ad4f55a414db9a509be49690612b9/users/a7b4d6e47dca438084fa880f3f2f8293/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.309259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6ad4f55a414db9a509be49690612b9, user_id=a7b4d6e47dca438084fa880f3f2f8293, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:52.312117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:52.326660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 807/1628] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 14:59:52 2022] PUT /identity/v3/projects/2c6ad4f55a414db9a509be49690612b9/users/a7b4d6e47dca438084fa880f3f2f8293/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:59:52.339000 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e186d0-8a7f-4262-a148-e3aa3156b4c5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.339288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e186d0-8a7f-4262-a148-e3aa3156b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.339563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e186d0-8a7f-4262-a148-e3aa3156b4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.401092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-64e186d0-8a7f-4262-a148-e3aa3156b4c5 None None] MFA Rules not processed for user `a7b4d6e47dca438084fa880f3f2f8293`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:52.404297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-64e186d0-8a7f-4262-a148-e3aa3156b4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:52.576248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 822/1629] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:52.585304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:52.607446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37247b-fc91-4866-a016-e4cf753e4195 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.607787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37247b-fc91-4866-a016-e4cf753e4195 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.608004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37247b-fc91-4866-a016-e4cf753e4195 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.609343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 823/1630] 10.222.0.61 () {54 vars in 912 bytes} [Sat Sep 24 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:59:52.619975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90512949-9f19-4f0d-aa80-5e0a18032b1d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.620276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90512949-9f19-4f0d-aa80-5e0a18032b1d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.620533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90512949-9f19-4f0d-aa80-5e0a18032b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.659583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-90512949-9f19-4f0d-aa80-5e0a18032b1d None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:52.664838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-90512949-9f19-4f0d-aa80-5e0a18032b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:52.744846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df31cbc-d9df-48a7-b7ab-05e221934c99 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:52.759892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:52.760529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.760813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.761088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.844465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 824/1631] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:59:52.848411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60be5d-5a9b-4e77-b343-f7bbe6f0a502 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.848805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60be5d-5a9b-4e77-b343-f7bbe6f0a502 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.849095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60be5d-5a9b-4e77-b343-f7bbe6f0a502 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.853980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 825/1632] 10.222.0.61 () {54 vars in 912 bytes} [Sat Sep 24 14:59:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:59:52.861026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:52.861629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:52.861983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:52.903472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:59:52.903875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:52.905344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:52.908891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:59:52.933131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 808/1633] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:52.947134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 826/1634] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 14:59:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:59:52.993662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:53.011835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd2c88-e72a-4e38-a2cf-1f7b39dfd2c2 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:53.016575 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:53.018510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:53.018979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:53.019407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:53.031284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:53.033095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:53.045482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 809/1635] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:53.766254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:53.784641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb5fba-b018-47a7-9a86-0c18a80748db None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:53.796910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43242e9a-1f02-46c1-b174-457d87ef4d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:53.797857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43242e9a-1f02-46c1-b174-457d87ef4d29 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:53.798295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43242e9a-1f02-46c1-b174-457d87ef4d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:53.799577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43242e9a-1f02-46c1-b174-457d87ef4d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:53.830236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43242e9a-1f02-46c1-b174-457d87ef4d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:53.833991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43242e9a-1f02-46c1-b174-457d87ef4d29 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:53.864691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 827/1636] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:59:59.563413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:59:59.573321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4a419-7648-4b26-980f-33fc4edfdc98 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:59:59.588693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:59:59.589347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:59:59.595209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:59:59.595543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:59:59.636799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:59:59.638309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:59:59.672878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 810/1637] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:00.088383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d663d-a43f-4b9b-bc39-b8bcd58ef440 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:00.088962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d663d-a43f-4b9b-bc39-b8bcd58ef440 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:00.088962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d663d-a43f-4b9b-bc39-b8bcd58ef440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:00.098350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:00.147373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-366d663d-a43f-4b9b-bc39-b8bcd58ef440 None None] MFA Rules not processed for user `9f74f51060554db18bc96199222e9261`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:00.148613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-366d663d-a43f-4b9b-bc39-b8bcd58ef440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:00.318716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 828/1638] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:00.334504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:00.381070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb0503-ecc4-47eb-8f77-4b7ffc579b71 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:00.390953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:00.391512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:00.391638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:00.394457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:00.405079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:00.407212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:00.419616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 811/1639] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:03.790446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594091bc-a08c-48a6-be2f-c6c765eb5651 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:03.791363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594091bc-a08c-48a6-be2f-c6c765eb5651 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:03.791805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594091bc-a08c-48a6-be2f-c6c765eb5651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:03.846870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-594091bc-a08c-48a6-be2f-c6c765eb5651 None None] MFA Rules not processed for user `cdbc898ffd4f4d5d87c3684fcc17313d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:03.848177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-594091bc-a08c-48a6-be2f-c6c765eb5651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:03.973390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 829/1640] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:03.999927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:04.017395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baa991-66c7-4778-9fcb-3c863846a3c7 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:04.036293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8d21d-733b-4af3-88cd-b05bde554638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:04.036293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8d21d-733b-4af3-88cd-b05bde554638 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:04.036293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8d21d-733b-4af3-88cd-b05bde554638 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:04.036617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8d21d-733b-4af3-88cd-b05bde554638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:04.051793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:04.056196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8d21d-733b-4af3-88cd-b05bde554638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:04.057847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8d21d-733b-4af3-88cd-b05bde554638 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:04.094054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 812/1641] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:05.005747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:05.006271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:05.006565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:05.074652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] MFA Rules not processed for user `5b63ee5b9ac44eb39617b291ac869db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:05.076468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:05.205732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 830/1642] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:05.217567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680d37-1951-4cbb-bbbd-9c7b03d85a4f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:05.218444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680d37-1951-4cbb-bbbd-9c7b03d85a4f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:05.218444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680d37-1951-4cbb-bbbd-9c7b03d85a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:05.279018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6a680d37-1951-4cbb-bbbd-9c7b03d85a4f None None] MFA Rules not processed for user `5b63ee5b9ac44eb39617b291ac869db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:05.280932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6a680d37-1951-4cbb-bbbd-9c7b03d85a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:05.307914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:05.325141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 813/1643] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:05.332392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47887b87-f3c6-42e2-a81d-83dab330bab5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:05.346405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca2e74-d914-4801-9d3f-30085c6b2b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:05.347339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca2e74-d914-4801-9d3f-30085c6b2b97 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:05.347793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca2e74-d914-4801-9d3f-30085c6b2b97 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:05.348251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca2e74-d914-4801-9d3f-30085c6b2b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:05.360745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca2e74-d914-4801-9d3f-30085c6b2b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:05.363849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca2e74-d914-4801-9d3f-30085c6b2b97 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:05.369974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:05.396171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 831/1644] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:05.617165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5d590-7992-4c1c-b26d-6845401d2d33 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:05.617604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5d590-7992-4c1c-b26d-6845401d2d33 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:05.618003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5d590-7992-4c1c-b26d-6845401d2d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:05.671606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-41a5d590-7992-4c1c-b26d-6845401d2d33 None None] MFA Rules not processed for user `db5a0bddb68047d68f0c2dd1af0cd57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:05.700404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-41a5d590-7992-4c1c-b26d-6845401d2d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:05.882033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 814/1645] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:05.883599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:05.883934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:05.884528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:05.948217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] MFA Rules not processed for user `db5a0bddb68047d68f0c2dd1af0cd57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:05.949466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:05.951374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:05.951768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:05.952164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:05.981497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 832/1646] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:06.002331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] MFA Rules not processed for user `9f74f51060554db18bc96199222e9261`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:06.004482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:06.018447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 815/1647] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:06.121364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:06.125462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:06.147617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab1a79-dcb4-4333-9a42-a5d639f0d1b5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:06.162644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c290-dce2-4bfd-bcdf-35574933040f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:06.163087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c290-dce2-4bfd-bcdf-35574933040f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.163369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c290-dce2-4bfd-bcdf-35574933040f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.163643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c290-dce2-4bfd-bcdf-35574933040f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.174265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c290-dce2-4bfd-bcdf-35574933040f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:06.176604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c290-dce2-4bfd-bcdf-35574933040f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:06.191513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 816/1648] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:06.243226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad33277-f25f-4bef-8a55-343f068df19e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:06.248693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:06.249344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.249637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.250202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] PATH_INFO: `/v3/users/cbc9c7ef2b4345e6bc3ec01935dc697b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.251399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc9c7ef2b4345e6bc3ec01935dc697b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:06.252987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 15:00:06.252987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 24 15:00:06.252987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: Sep 24 15:00:06.254542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:06.432320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc9c7ef2b4345e6bc3ec01935dc697b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:06.448566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc9c7ef2b4345e6bc3ec01935dc697b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:06.475510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc9c7ef2b4345e6bc3ec01935dc697b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:06.476512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc9c7ef2b4345e6bc3ec01935dc697b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:06.500905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc9c7ef2b4345e6bc3ec01935dc697b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:06.513944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 833/1649] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:06 2022] DELETE /identity/v3/users/cbc9c7ef2b4345e6bc3ec01935dc697b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:06.518785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622daba-cdcd-49a6-ae27-e7f6d54f8a58 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.519287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622daba-cdcd-49a6-ae27-e7f6d54f8a58 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.519380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622daba-cdcd-49a6-ae27-e7f6d54f8a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.545928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5622daba-cdcd-49a6-ae27-e7f6d54f8a58 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:06.546499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5622daba-cdcd-49a6-ae27-e7f6d54f8a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:06.660517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 817/1650] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:06.675504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:06.680910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:06.768060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff7f6-2a24-4de7-b1a7-6fbc0b7a075a None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.768542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff7f6-2a24-4de7-b1a7-6fbc0b7a075a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.768954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff7f6-2a24-4de7-b1a7-6fbc0b7a075a None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.771152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 818/1651] 10.222.0.61 () {54 vars in 912 bytes} [Sat Sep 24 15:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 15:00:06.780687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.782299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.782825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.835218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c286fd-e624-488f-ab06-82bdf295d5d2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:06.844620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf8cf2-acb1-45a7-9967-89be957fd9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:06.845294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf8cf2-acb1-45a7-9967-89be957fd9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.845596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf8cf2-acb1-45a7-9967-89be957fd9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.845869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf8cf2-acb1-45a7-9967-89be957fd9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.865454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf8cf2-acb1-45a7-9967-89be957fd9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:06.867081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf8cf2-acb1-45a7-9967-89be957fd9a7 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:06.878297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:06.878651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 834/1652] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:06.879519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:06.880088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f09d-a80e-4cc8-ba89-d20efb5e6a8d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.880457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f09d-a80e-4cc8-ba89-d20efb5e6a8d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.880647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f09d-a80e-4cc8-ba89-d20efb5e6a8d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.882203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 835/1653] 10.222.0.61 () {54 vars in 912 bytes} [Sat Sep 24 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:00:06.889723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.890069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.890312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.903916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 819/1654] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:06.927992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:06.929418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:06.938159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:06.940162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 836/1655] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:06.944118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:06.962821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32a95-9e99-4769-97ca-b8906c4fa8c2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:06.971129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:06.975557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:06.976568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:06.977128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:06.978264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:06.984215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f435bf4-76df-428e-8496-274bf43e6d06 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:06.995046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:06.998386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:07.004500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:07.005109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:07.005381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:07.005655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:07.017633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:07.024532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:07.024668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 820/1656] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:07.027859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:07.049026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 837/1657] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:07.818005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:07.819966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:07.851160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147c51f-6aad-4337-8603-fa3c8edcbfa7 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:07.860942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:07.862656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:07.864139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:07.864583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] PATH_INFO: `/v3/projects/c2e445f0acdd4718a39a8aef7043802b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:07.876505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e445f0acdd4718a39a8aef7043802b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:07.877886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 15:00:07.877886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 24 15:00:07.877886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: Sep 24 15:00:07.883675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:08.001733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:08.068853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:08.070999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:08.118279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e445f0acdd4718a39a8aef7043802b', 'request_id': 'req-4d21b545-6f50-435e-8db3-b19162cf09e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:08.132945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e445f0acdd4718a39a8aef7043802b', 'request_id': 'req-4d21b545-6f50-435e-8db3-b19162cf09e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:08.169521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e445f0acdd4718a39a8aef7043802b', 'request_id': 'req-4d21b545-6f50-435e-8db3-b19162cf09e6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:08.198018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] The token cache is being invalidate because project c2e445f0acdd4718a39a8aef7043802b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:08.198499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:08.200316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4d21b545-6f50-435e-8db3-b19162cf09e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:08.206341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 821/1658] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:07 2022] DELETE /identity/v3/projects/c2e445f0acdd4718a39a8aef7043802b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:08.208744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3c62-5963-4a07-97d1-ccb45811eafb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.210107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f20d42-c613-4f71-8566-58a741342849 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:08.210465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3c62-5963-4a07-97d1-ccb45811eafb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.210885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3c62-5963-4a07-97d1-ccb45811eafb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.216031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 822/1659] 10.222.0.61 () {54 vars in 961 bytes} [Sat Sep 24 15:00:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:00:08.217225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd68fc8-97b9-421e-b482-dd78cad66e21 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.219210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd68fc8-97b9-421e-b482-dd78cad66e21 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.219640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd68fc8-97b9-421e-b482-dd78cad66e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.255024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9cd68fc8-97b9-421e-b482-dd78cad66e21 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:08.264104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9cd68fc8-97b9-421e-b482-dd78cad66e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:08.344366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:08.345081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.345667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.346046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.348008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:08.370359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:08.378571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa669bf-4766-49d0-bf79-b74aeecf61e7 service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:08.409574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 838/1660] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:08.415659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8faf12-1304-4348-9840-1e565ada22fb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.416270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8faf12-1304-4348-9840-1e565ada22fb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.416988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8faf12-1304-4348-9840-1e565ada22fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.418247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 823/1661] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:08.419830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812e742-28d1-4dfd-8052-970732214552 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.420398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812e742-28d1-4dfd-8052-970732214552 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.420772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812e742-28d1-4dfd-8052-970732214552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.468541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0812e742-28d1-4dfd-8052-970732214552 None None] MFA Rules not processed for user `cdbc898ffd4f4d5d87c3684fcc17313d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:08.470054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7b8faf12-1304-4348-9840-1e565ada22fb None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:08.471614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7b8faf12-1304-4348-9840-1e565ada22fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:08.477294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0812e742-28d1-4dfd-8052-970732214552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:08.500239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 839/1662] 10.222.0.61 () {58 vars in 1058 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:08.507310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.507597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.507869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.547176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:08.549090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:08.571991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 840/1663] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:08.576085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:08.605945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c820288-bb42-4b9b-a6c0-9b96e857c896 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:08.621682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:08.622603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.623020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.623427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.625300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 824/1664] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:08.626386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812e742-28d1-4dfd-8052-970732214552 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:08.649099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:08.650766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:08.656385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812e742-28d1-4dfd-8052-970732214552 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:08.662796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:08.663780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.664077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.664487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.665170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:08.665479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:08.667973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:08.687670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 841/1665] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:08.688441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 825/1666] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:08.694325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:08.731684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c988d-acad-40a8-a5cb-3182ac4d09ff service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:08.752112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0d95-f88c-4000-8564-2dedfd2320e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:08.752353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0d95-f88c-4000-8564-2dedfd2320e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.752580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0d95-f88c-4000-8564-2dedfd2320e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.753185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0d95-f88c-4000-8564-2dedfd2320e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:08.753492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d0d95-f88c-4000-8564-2dedfd2320e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:08.761337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d0d95-f88c-4000-8564-2dedfd2320e5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:08.867001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 842/1667] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:08.868150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:08.893010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef32c5-4025-4a0d-9f3b-5e68934b3b6b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:08.993436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:08.993755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:08.995318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:09.023573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] MFA Rules not processed for user `db5a0bddb68047d68f0c2dd1af0cd57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:09.024904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:09.035340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:09.107080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:09.107712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:09.110125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:09.110560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:09.111812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:09.114407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:09.141684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:09.152110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 826/1668] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/users => generated 284 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:09.158599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:09.187825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c952f9-db58-47f3-a30c-84ac9dea1d8b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:09.199357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:09.200072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:09.210868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 843/1669] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:09.211072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:09.211072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:09.211072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:09.214046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:09.252118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 827/1670] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:09.253066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:09.279686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51a597-1005-44ef-90e0-94e8ca432d7b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:09.284938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1794eb-d12e-4338-9c48-01c4c9312fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:09.285757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1794eb-d12e-4338-9c48-01c4c9312fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:09.286075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1794eb-d12e-4338-9c48-01c4c9312fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:09.286350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1794eb-d12e-4338-9c48-01c4c9312fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:09.287005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1794eb-d12e-4338-9c48-01c4c9312fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:09.288628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1794eb-d12e-4338-9c48-01c4c9312fa9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:09.324917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 844/1671] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:09.326358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:09.396052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca3d49-c879-4fcc-a0d8-bfd0693d4e8f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:09.416470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:09.417155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:09.417437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:09.417712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] PATH_INFO: `/v3/projects/7863acf80f90418a8bad723ecd01b6de/users/457618b1db6446518997d85470064f75/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:09.511616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7863acf80f90418a8bad723ecd01b6de, user_id=457618b1db6446518997d85470064f75, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:09.514214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:09.553269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 828/1672] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:09 2022] PUT /identity/v3/projects/7863acf80f90418a8bad723ecd01b6de/users/457618b1db6446518997d85470064f75/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:09.565211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:09.565752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:09.566196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:09.622921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] MFA Rules not processed for user `457618b1db6446518997d85470064f75`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:09.624175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:09.785244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 845/1673] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:09.789541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:10.054311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464be007-25f3-4e58-9fb9-84ac1eae51c5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:10.067351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:10.069925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.070372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.071231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.083802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:10.086926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:10.168376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 829/1674] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:09 2022] POST /identity/v3/projects => generated 367 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:10.170598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:10.188543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809864a-cca0-4de5-bbf0-9c406570c566 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:10.273553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:10.274695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.275186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.275641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.276663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:10.279129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:10.327536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 846/1675] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:10 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:10.331613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:10.347731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cb40a-f111-4595-8043-d6cb02631f83 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:10.361790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:10.362724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.363630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.364737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.366120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:10.370706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:10.397483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 830/1676] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:10.401459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:10.417449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcae8d6-4c48-41d5-8d69-79e47e483dae admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:10.430413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:10.431246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.431619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.432120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] PATH_INFO: `/v3/projects/9d7167b77f6140e1bb8ed3488b36acfe/users/74c591543d104e2db86c572ab7d896f9/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.469769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7167b77f6140e1bb8ed3488b36acfe, user_id=74c591543d104e2db86c572ab7d896f9, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:10.474697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:10.492906 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 847/1677] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:10 2022] PUT /identity/v3/projects/9d7167b77f6140e1bb8ed3488b36acfe/users/74c591543d104e2db86c572ab7d896f9/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:10.496933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:10.510068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c7b26-466d-4b88-9e56-6d364b26d697 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:10.689207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308329b2-2967-49ad-a89a-498dd638e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:10.694060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308329b2-2967-49ad-a89a-498dd638e9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.694461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308329b2-2967-49ad-a89a-498dd638e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.694618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308329b2-2967-49ad-a89a-498dd638e9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.695373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308329b2-2967-49ad-a89a-498dd638e9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:10.697206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308329b2-2967-49ad-a89a-498dd638e9a3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:10.742908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 831/1678] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:10.747803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:10.783453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5206c66-296e-41d7-b825-611e166f8351 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:10.819247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:10.820149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.820415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.820883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] PATH_INFO: `/v3/projects/9d7167b77f6140e1bb8ed3488b36acfe/users/74c591543d104e2db86c572ab7d896f9/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.827303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7167b77f6140e1bb8ed3488b36acfe, user_id=74c591543d104e2db86c572ab7d896f9, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:10.830094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:10.868234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 848/1679] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:10 2022] PUT /identity/v3/projects/9d7167b77f6140e1bb8ed3488b36acfe/users/74c591543d104e2db86c572ab7d896f9/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:10.878803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:10.879092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:10.879370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:10.922520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:10.969681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] MFA Rules not processed for user `74c591543d104e2db86c572ab7d896f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:10.971576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:11.053328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:11.054494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:11.119679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:11.180687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 832/1680] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:11.195109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:11.202706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:11.290262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a92d-c243-4359-a1ec-d446d2fd9a54 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:11.306213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:11.307938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:11.308399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:11.309204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] PATH_INFO: `/v3/users/3f880497e66e4a94bb2b81ef5fdc4a72` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:11.312021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f880497e66e4a94bb2b81ef5fdc4a72)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:11.327647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:11.334999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bde2ec-1ac7-44d2-aa64-6a17bcfe7d36 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:11.343437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fecdea-1f90-4128-b43b-abdfeec58ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:11.344592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fecdea-1f90-4128-b43b-abdfeec58ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:11.347382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fecdea-1f90-4128-b43b-abdfeec58ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:11.348395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fecdea-1f90-4128-b43b-abdfeec58ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:11.370739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fecdea-1f90-4128-b43b-abdfeec58ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:11.374560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fecdea-1f90-4128-b43b-abdfeec58ca1 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:11.419377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 833/1681] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:11.446271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f880497e66e4a94bb2b81ef5fdc4a72'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:11.460926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f880497e66e4a94bb2b81ef5fdc4a72'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:11.513604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f880497e66e4a94bb2b81ef5fdc4a72'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:11.514438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f880497e66e4a94bb2b81ef5fdc4a72'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:11.533411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f880497e66e4a94bb2b81ef5fdc4a72'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:11.551659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 849/1682] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:11 2022] DELETE /identity/v3/users/3f880497e66e4a94bb2b81ef5fdc4a72 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:11.554132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:11.554627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:11.554914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:11.619729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:11.628235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:11.713830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:11.715527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:11.838269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 834/1683] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:11.864369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:11.874187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:11.962749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd0a91-e920-4bf2-8a65-3eefea9ecc4e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:11.977055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:11.980913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:11.981225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:11.981532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:11.981816 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:11.988940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:12.016886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8648cc-be9f-40bd-a1d1-c36738ee8453 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:12.022442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:12.024535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:12.025031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:12.025487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:12.037619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:12.039693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:12.056682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 835/1684] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:12.090513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:12.117730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:12.119854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf6ad6-f944-4d32-b6b7-cb74ba4ebf3d service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:12.141833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 850/1685] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:13.223190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:13.226127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:13.259929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f225a53-98b3-466f-a7de-07b54715b917 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:13.265845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:13.266700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:13.267359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:13.267850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] PATH_INFO: `/v3/projects/727eb826972a49cf87453855388045f6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:13.270543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=727eb826972a49cf87453855388045f6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:13.273171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:13.412737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727eb826972a49cf87453855388045f6', 'request_id': 'req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:13.417078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee80a39-f390-4543-894c-cb17fbda89ce None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:13.417589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee80a39-f390-4543-894c-cb17fbda89ce None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:13.417676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee80a39-f390-4543-894c-cb17fbda89ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:13.436641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727eb826972a49cf87453855388045f6', 'request_id': 'req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:13.453325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727eb826972a49cf87453855388045f6', 'request_id': 'req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:13.484293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] The token cache is being invalidate because project 727eb826972a49cf87453855388045f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:13.485294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:13.485773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36d141fb-6c0c-4854-8b08-ec6c7ae6de6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:13.491213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 836/1686] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:13 2022] DELETE /identity/v3/projects/727eb826972a49cf87453855388045f6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:13.500932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:13.501235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:13.502476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:13.529701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fee80a39-f390-4543-894c-cb17fbda89ce None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:13.540958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fee80a39-f390-4543-894c-cb17fbda89ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:13.573740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:13.580819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:13.637370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 851/1687] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:00:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:13.745168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 837/1688] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:13.754579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:13.755153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:13.755652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:13.791137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:13.792568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:13.816902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 852/1689] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:13.818435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:13.836959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea6223-a2fb-40a3-9622-8117489b3f06 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:13.848427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:13.849409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:13.849930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:13.851023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:13.852930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:13.854843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:13.863359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:13.906501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 838/1690] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:13.909132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:14.008458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00c360-7a77-4e30-99b6-33ea04a2b277 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:14.014611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:14.016824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:14.022533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:14.024314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.024865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.026096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.026838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:14.029861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:14.093714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 853/1691] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:13 2022] POST /identity/v3/projects => generated 403 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:14.109117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:14.227232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:14.378922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094913-4be2-4dd5-b7c4-fe2911142caf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:14.402518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03b55-34ca-4870-9fe8-8b2b69bde27b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:14.402833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:14.403478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.403763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.404047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.404880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:14.406898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:14.482037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 854/1692] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:14 2022] POST /identity/v3/users => generated 302 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:14.483055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:14.511200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e426b-38e6-4213-a37e-5d13dc519b80 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:14.518005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:14.518628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.518911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.519185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.519840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:14.521465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:14.556334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 855/1693] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:14.561942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:14.586078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c405c0-df95-41cd-8682-7a47680a7f08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:14.586078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c405c0-df95-41cd-8682-7a47680a7f08 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.586078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c405c0-df95-41cd-8682-7a47680a7f08 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.586078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c405c0-df95-41cd-8682-7a47680a7f08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.591386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7b72a-c481-41d6-bced-bffe6a0447b1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:14.629099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:14.629729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.630040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.630318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.638758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:14.649098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:14.652245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:14.697041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 856/1694] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:14.698031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:14.747556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9794e16-13c3-4a35-8e5c-6eb41e81ef05 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:14.781758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:14.781758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.781758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.781758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] PATH_INFO: `/v3/projects/5f3a7dac18ba421d9ff81fb4665419df/users/f995a036c8b643fe9d182b925c34be97/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.806325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c405c0-df95-41cd-8682-7a47680a7f08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:14.809498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3a7dac18ba421d9ff81fb4665419df, user_id=f995a036c8b643fe9d182b925c34be97, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:14.809707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c405c0-df95-41cd-8682-7a47680a7f08 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:14.812694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:14.834234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 839/1695] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:00:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 15:00:14.839914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 857/1696] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:14 2022] PUT /identity/v3/projects/5f3a7dac18ba421d9ff81fb4665419df/users/f995a036c8b643fe9d182b925c34be97/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:14.844949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5f80f-6fdf-49c6-a58f-cd1fb3711887 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:14.845346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5f80f-6fdf-49c6-a58f-cd1fb3711887 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:14.845486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5f80f-6fdf-49c6-a58f-cd1fb3711887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:14.883073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4ed5f80f-6fdf-49c6-a58f-cd1fb3711887 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:14.898204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ed5f80f-6fdf-49c6-a58f-cd1fb3711887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:15.069082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 840/1697] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:15.093471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:15.095894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:15.255229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fbd21-28b9-4e5e-96ee-227e1a699bf5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:15.261833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:15.262497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:15.262874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:15.263053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:15.283662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:15.285251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:15.328806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 858/1698] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:16.859391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:16.859907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:16.860076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:16.904368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:16.909779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] MFA Rules not processed for user `f995a036c8b643fe9d182b925c34be97`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:16.911791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:17.043345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 841/1699] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:17.044238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:17.060629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098f0d1-feac-40f1-85f8-1e6c359b8055 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:17.074653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:17.075484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:17.075913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:17.076338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:17.077125 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:17.079273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:17.145179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 859/1700] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:17 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:17.149896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:17.176645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee6bdf-e9a1-4edb-95c2-c1ca8e6100c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:17.296889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:17.297556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:17.302117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:17.302733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:17.308153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:17.313175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:17.369771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 842/1701] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:17 2022] POST /identity/v3/users => generated 308 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:17.371378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:17.402117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0d55a-4c82-4b82-ab2b-9f34cf68e97a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:17.415786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:17.416563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:17.416914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:17.417250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:17.418015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:17.419967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:17.469230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 860/1702] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:17.473662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:17.517352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e4564-d82f-4da4-9e90-a53c3d338982 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:17.526622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:17.527799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:17.528690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:17.529942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] PATH_INFO: `/v3/projects/8031e2a2c56343c3981c76799d519147/users/d4e5c64295ca4c459571387891260895/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:17.550352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8031e2a2c56343c3981c76799d519147, user_id=d4e5c64295ca4c459571387891260895, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:17.556403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:17.616601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 843/1703] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:17 2022] PUT /identity/v3/projects/8031e2a2c56343c3981c76799d519147/users/d4e5c64295ca4c459571387891260895/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:17.620450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:17.655994 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efc41-726e-4f87-8f76-f38819124e2c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:17.747429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110b6c9-711e-4647-91fb-8c8bba93b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:17.748381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110b6c9-711e-4647-91fb-8c8bba93b5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:17.748829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110b6c9-711e-4647-91fb-8c8bba93b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:17.749337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110b6c9-711e-4647-91fb-8c8bba93b5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:17.750108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110b6c9-711e-4647-91fb-8c8bba93b5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:17.751987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110b6c9-711e-4647-91fb-8c8bba93b5e9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:17.765428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 861/1704] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:17.769611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:17.792563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4b118-1493-4cf7-b73a-54bee2046a2c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:17.805839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:17.809503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:17.811275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:17.811751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] PATH_INFO: `/v3/projects/8031e2a2c56343c3981c76799d519147/users/d4e5c64295ca4c459571387891260895/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:17.816764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8031e2a2c56343c3981c76799d519147, user_id=d4e5c64295ca4c459571387891260895, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:17.827119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:17.854299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 844/1705] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:17 2022] PUT /identity/v3/projects/8031e2a2c56343c3981c76799d519147/users/d4e5c64295ca4c459571387891260895/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:18.040635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:18.041060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:18.041462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:18.096933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:18.099500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:18.165465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:18.224899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 862/1706] 10.222.0.72 () {58 vars in 1015 bytes} [Sat Sep 24 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:18.607281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:18.609305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:18.629267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:18.651587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41447aee-2ee0-4035-b09a-8b552ac39a74 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:18.674341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:18.685535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:18.685823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:18.686143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:18.761805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:18.763324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:18.812325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 863/1707] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:18.813309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:18.818430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:18.861200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979d09-cd0e-46e2-bc2d-64c1f5feec21 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:18.889147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b170ab9-12e7-4b1b-b2bf-e3ec7f252658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:18.889771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:18.889895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b170ab9-12e7-4b1b-b2bf-e3ec7f252658 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:18.890069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b170ab9-12e7-4b1b-b2bf-e3ec7f252658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:18.890324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b170ab9-12e7-4b1b-b2bf-e3ec7f252658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:18.921939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e54a3-3d0c-4134-8b3d-2ef6c95ea04f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:18.926508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b170ab9-12e7-4b1b-b2bf-e3ec7f252658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:18.927969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b170ab9-12e7-4b1b-b2bf-e3ec7f252658 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:18.945601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:18.947259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:18.952985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:18.953555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] PATH_INFO: `/v3/users/634fd179e81745758f35c9f6fd6c9b9e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:18.955897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 864/1708] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:18.957488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=634fd179e81745758f35c9f6fd6c9b9e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:18.969293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:19.113700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634fd179e81745758f35c9f6fd6c9b9e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.125062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634fd179e81745758f35c9f6fd6c9b9e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.144494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634fd179e81745758f35c9f6fd6c9b9e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.145633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634fd179e81745758f35c9f6fd6c9b9e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.156513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634fd179e81745758f35c9f6fd6c9b9e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.163982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 845/1709] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:18 2022] DELETE /identity/v3/users/634fd179e81745758f35c9f6fd6c9b9e => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:19.168419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55112cfd-fc20-44cf-a10a-7be8af0568bb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:19.168900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55112cfd-fc20-44cf-a10a-7be8af0568bb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:19.168982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55112cfd-fc20-44cf-a10a-7be8af0568bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:19.208176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-55112cfd-fc20-44cf-a10a-7be8af0568bb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:19.209666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55112cfd-fc20-44cf-a10a-7be8af0568bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:19.379882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:19.381565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:19.407039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 865/1710] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:19.551797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426116af-6487-44a5-82e5-824c40cc34d4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:19.577875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:19.577875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:19.577875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:19.577875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] PATH_INFO: `/v3/users/457618b1db6446518997d85470064f75` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:19.586467 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=457618b1db6446518997d85470064f75)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:19.592313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:19.714567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457618b1db6446518997d85470064f75'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.734158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457618b1db6446518997d85470064f75'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.771380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457618b1db6446518997d85470064f75'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.772393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457618b1db6446518997d85470064f75'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.787085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:19.793995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457618b1db6446518997d85470064f75'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:19.802118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 846/1711] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:19 2022] DELETE /identity/v3/users/457618b1db6446518997d85470064f75 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:19.811368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:19.814426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:19.814719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:19.859486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:19.865340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:19.911328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:19.913049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:20.093202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 866/1712] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:20.096956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817d380-0267-496c-a684-0321fdac6b3d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:20.107264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:20.107916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:20.108201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:20.111178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:20.121224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:20.167342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:20.184940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:20.221335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 847/1713] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:20.259819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:20.261436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:20.274873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e90561-6076-4015-833e-c84c7fb64114 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:20.279247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:20.280025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:20.280454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:20.280880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:20.294042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:20.296246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:20.308288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 867/1714] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:21.014350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:21.015866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:21.028685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c663d-93d5-4166-98dc-5f30e5ae92f2 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:21.033557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:21.035600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.036118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.036599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] PATH_INFO: `/v3/projects/7863acf80f90418a8bad723ecd01b6de` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.038654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7863acf80f90418a8bad723ecd01b6de)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:21.041951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:21.115886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7863acf80f90418a8bad723ecd01b6de', 'request_id': 'req-f931e055-d597-4a19-83d1-f9c2a31881aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.122093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7863acf80f90418a8bad723ecd01b6de', 'request_id': 'req-f931e055-d597-4a19-83d1-f9c2a31881aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.129992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7863acf80f90418a8bad723ecd01b6de', 'request_id': 'req-f931e055-d597-4a19-83d1-f9c2a31881aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.145009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] The token cache is being invalidate because project 7863acf80f90418a8bad723ecd01b6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:21.145364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.146140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f931e055-d597-4a19-83d1-f9c2a31881aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.150768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 848/1715] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:21 2022] DELETE /identity/v3/projects/7863acf80f90418a8bad723ecd01b6de => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:21.151926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:21.163388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62781b2-c894-439a-86e5-b89774f78ffa service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:21.178813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97033-8a27-4bb1-b1d8-03a531bcbde1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.179620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97033-8a27-4bb1-b1d8-03a531bcbde1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.180166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97033-8a27-4bb1-b1d8-03a531bcbde1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.183938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 849/1716] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:00:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:00:21.193925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.194439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.195285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.255947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:21.256547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:21.258059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:21.281605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:21.284851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.286161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.287135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] PATH_INFO: `/v3/users/74c591543d104e2db86c572ab7d896f9` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.289130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c591543d104e2db86c572ab7d896f9)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:21.292069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:21.382067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c591543d104e2db86c572ab7d896f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.393109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c591543d104e2db86c572ab7d896f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.414785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c591543d104e2db86c572ab7d896f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.416119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 850/1717] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:00:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:21.418171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c591543d104e2db86c572ab7d896f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.423896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:21.425622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:21.431694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a116b61a-414f-4931-8583-196e760d99e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c591543d104e2db86c572ab7d896f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:21.440030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e2173-7a86-4148-b83a-b716d6c888eb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:21.441955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 868/1718] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:21 2022] DELETE /identity/v3/users/74c591543d104e2db86c572ab7d896f9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:21.449267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:21.549728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:21.550711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.550969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.551673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.553766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:21.571482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:21.573259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:21.613989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 851/1719] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:21.731183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.731476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.731966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.779869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:21.781376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:21.833623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] MFA Rules not processed for user `d4e5c64295ca4c459571387891260895`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:21.835645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:21.923786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52086b-ba76-4cc6-92d3-49216d92d32c service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:21.930023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:21.930915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:21.931202 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:21.931479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] PATH_INFO: `/v3/projects/da06115b1d3a4938824cc4c1b2460454` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:21.932637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=da06115b1d3a4938824cc4c1b2460454)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:21.934866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:21.997002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 869/1720] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:22.002151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.002151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.032464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0a9d9-326b-4b3c-a633-6b760db2d7a4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.080619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da06115b1d3a4938824cc4c1b2460454', 'request_id': 'req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.098079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da06115b1d3a4938824cc4c1b2460454', 'request_id': 'req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.120980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:22.123071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da06115b1d3a4938824cc4c1b2460454', 'request_id': 'req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.131609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.132283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.132749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.133006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] PATH_INFO: `/v3/projects/9d7167b77f6140e1bb8ed3488b36acfe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.135521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d7167b77f6140e1bb8ed3488b36acfe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.153096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.160092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] The token cache is being invalidate because project da06115b1d3a4938824cc4c1b2460454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:22.162804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.164572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4ed3dde1-706c-4872-8aa8-8bbde5263cfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.166507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 852/1721] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:21 2022] DELETE /identity/v3/projects/da06115b1d3a4938824cc4c1b2460454 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:22.175447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401286-f051-49a5-8530-3d1de37dc1c8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.176283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401286-f051-49a5-8530-3d1de37dc1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.177097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401286-f051-49a5-8530-3d1de37dc1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.219243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-65401286-f051-49a5-8530-3d1de37dc1c8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:22.224991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-65401286-f051-49a5-8530-3d1de37dc1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.251622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 853/1722] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:22.259339 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.259631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.259910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.268828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d7167b77f6140e1bb8ed3488b36acfe', 'request_id': 'req-f526541b-3ddf-4219-a83f-b0b346c3228c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.294156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d7167b77f6140e1bb8ed3488b36acfe', 'request_id': 'req-f526541b-3ddf-4219-a83f-b0b346c3228c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.297273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:22.300078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.303855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d7167b77f6140e1bb8ed3488b36acfe', 'request_id': 'req-f526541b-3ddf-4219-a83f-b0b346c3228c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.330233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] The token cache is being invalidate because project 9d7167b77f6140e1bb8ed3488b36acfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:22.330698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.331518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f526541b-3ddf-4219-a83f-b0b346c3228c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:22.332424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:22.335272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 870/1723] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:21 2022] DELETE /identity/v3/projects/9d7167b77f6140e1bb8ed3488b36acfe => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:22.340926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.341615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.342137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.375070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:22.376457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.431938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 854/1724] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:22.433212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.445384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd8c37-2768-43c3-abd5-810126a5c258 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.457297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.457955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.458232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.460717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.461402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:22.463927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.468158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df27e-fe64-4700-9b8e-c439756daf4c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.489098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 855/1725] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:22.490418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de69253-a3b1-431e-a0ad-b48d04ad364d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.490644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 871/1726] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:22.490767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de69253-a3b1-431e-a0ad-b48d04ad364d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.490968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de69253-a3b1-431e-a0ad-b48d04ad364d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.492410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 856/1727] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:00:22.496668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.499908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.510443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.510916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf275c82-0d42-4fbf-9363-afd2a1718a27 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.517435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.518812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea6d4b-4b46-4fe2-b9c9-5d18ddc1c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.519686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6d4b-4b46-4fe2-b9c9-5d18ddc1c92a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.520062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6d4b-4b46-4fe2-b9c9-5d18ddc1c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.520563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6d4b-4b46-4fe2-b9c9-5d18ddc1c92a admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.524329 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6d4b-4b46-4fe2-b9c9-5d18ddc1c92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.524329 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6d4b-4b46-4fe2-b9c9-5d18ddc1c92a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.553281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:22.554936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.561440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 872/1728] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:22.561440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.561440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.561440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.594909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:22.596281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.658577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 857/1729] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:22.659369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.682816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d3ed9-e6ae-455a-b693-b5d59dc2d3d7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.692284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.692284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.692284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.692284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.692761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.694619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.732290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 858/1730] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:22.736145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.750496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:22.770636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae365a-7419-429a-a191-5da6ba8457f9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.778289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 873/1731] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:22.779641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.780520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.782976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.783270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.783544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.785050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:22.786527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.792104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03998f-cf34-444d-a5d2-9409f875248d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.794679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.796877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.797409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.797758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.798687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.799781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.801126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.822221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 874/1732] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:22.823862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.825504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:22.832443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 859/1733] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:22.834047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.847432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f346-9f0a-4987-b29b-6cd7d4ba55a1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.855652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38792335-9106-4e40-a854-7acf02581841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.855791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38792335-9106-4e40-a854-7acf02581841 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.856166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38792335-9106-4e40-a854-7acf02581841 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.856539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38792335-9106-4e40-a854-7acf02581841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.869764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38792335-9106-4e40-a854-7acf02581841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.871791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38792335-9106-4e40-a854-7acf02581841 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.872907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f78a4-346a-4729-87fd-38d0d0de97d2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.885068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.885720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.886048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 875/1734] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:22.886536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.886887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.887615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.894573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.899885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38792335-9106-4e40-a854-7acf02581841 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.913721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38792335-9106-4e40-a854-7acf02581841 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.935975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 860/1735] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:22.940183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:22.951452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.954819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.955314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.955614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:22.956291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:22.960610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:22.964187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146c6f9-0704-42dc-b302-e04c3c9f62a7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:22.980162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547d4cb-09b2-4c2a-974b-8bf7f45241dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:22.980855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547d4cb-09b2-4c2a-974b-8bf7f45241dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:22.981138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547d4cb-09b2-4c2a-974b-8bf7f45241dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:22.981420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547d4cb-09b2-4c2a-974b-8bf7f45241dd admin admin] PATH_INFO: `/v3/projects/e492221e865242078e186f68f9ebd96f/users/f5722a9f26114f33a97df455364f355e/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.003580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547d4cb-09b2-4c2a-974b-8bf7f45241dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e492221e865242078e186f68f9ebd96f, user_id=f5722a9f26114f33a97df455364f355e, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.008091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547d4cb-09b2-4c2a-974b-8bf7f45241dd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.027554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 876/1736] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:22 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:23.036233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.061415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 861/1737] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:22 2022] PUT /identity/v3/projects/e492221e865242078e186f68f9ebd96f/users/f5722a9f26114f33a97df455364f355e/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:23.073726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.074056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.074334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.125920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] MFA Rules not processed for user `f5722a9f26114f33a97df455364f355e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:23.127308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:23.148617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0abde-733d-4951-8ef0-0ac3d51bec96 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.166537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:23.219543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 862/1738] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:23.223217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.236379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.237142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.237429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.237710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.238658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.240806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.270127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 877/1739] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:23 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:23.274231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.304250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b8adb-2281-4c90-8b62-ade71c89bab5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.310566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.311184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.311469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.311753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.312409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.314258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.342027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc14221-74e0-4fe4-96f8-6fb314bf2ab8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.345705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 878/1740] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:23.349926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.358637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.359419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.359858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.360240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.361524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.363736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc3894-30c4-4c6e-aa11-822c478258e3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.364719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.375811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.379722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.379722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.379722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.379722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.380184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.399790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 879/1741] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:23.403932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.412410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 863/1742] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:23 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:23.418514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5740d-a36e-4fc9-b4c5-88c8041f739d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.420065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.480388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:23.607113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59c44b-df78-4404-abdf-c4219af66404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.608099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c44b-df78-4404-abdf-c4219af66404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.608388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c44b-df78-4404-abdf-c4219af66404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.608663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c44b-df78-4404-abdf-c4219af66404 admin admin] PATH_INFO: `/v3/projects/181ad9c9850742ea814a39b97e61c79b/users/2b7e96d63d7446cfa3bc80784944c474/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.637286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebc409-8cb4-40eb-9826-2505ff655d07 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.650651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59c44b-df78-4404-abdf-c4219af66404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=181ad9c9850742ea814a39b97e61c79b, user_id=2b7e96d63d7446cfa3bc80784944c474, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.653635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59c44b-df78-4404-abdf-c4219af66404 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.660144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc4342-33f9-4ee3-8481-601382b76de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.661031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4342-33f9-4ee3-8481-601382b76de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.661643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4342-33f9-4ee3-8481-601382b76de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.662149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4342-33f9-4ee3-8481-601382b76de4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.663027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc4342-33f9-4ee3-8481-601382b76de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.667383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc4342-33f9-4ee3-8481-601382b76de4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.691855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 880/1743] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:23 2022] PUT /identity/v3/projects/181ad9c9850742ea814a39b97e61c79b/users/2b7e96d63d7446cfa3bc80784944c474/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:23.693792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eec313-14d3-45e8-bfeb-4b4ac5db955f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.694560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eec313-14d3-45e8-bfeb-4b4ac5db955f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.695250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eec313-14d3-45e8-bfeb-4b4ac5db955f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.715046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 881/1745] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:00:23 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:00:23.715245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 864/1745] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:23 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:23.716527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.717602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.717986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.718742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.720673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.722035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.749883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:23.751451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:23.797616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:23.811591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:23.832040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:23.864177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 865/1746] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:23.865167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.878580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60e218-474d-42a8-a32e-62295dcf81ab None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:23.885072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:23.886099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:23.886582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:23.887047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:23.887872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:23.889668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:23.906980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 866/1747] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:23.909946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:23.911899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:24.022043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 882/1748] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:24.023068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:24.037139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d032e2c-6b50-40d3-b92a-17a19afbe594 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:24.038638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33635ecc-1bcd-4971-af77-3f9e82ed3c7e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:24.041557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:24.042427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:24.042867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:24.042971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:24.044576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:24.045577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:24.046097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:24.046471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:24.047393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:24.051593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:24.086733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:24.088270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:24.090498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 883/1749] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:24.100980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:24.108427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 867/1750] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:24.109265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:24.111345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:24.130765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821b80-4831-423b-89b8-fa323a8c930d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:24.153747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:24.154940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:24.155444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:24.156023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] PATH_INFO: `/v3/projects/4c101c75d12f452b99424cd38040ab40/users/fffdc9c626c14623950518dfc3e948d8/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:24.164641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b06500-e478-4c72-b066-bac1d1caf7f5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:24.183397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf94b1c-dc13-4262-9344-50882626ff1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:24.186154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94b1c-dc13-4262-9344-50882626ff1c service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:24.186621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94b1c-dc13-4262-9344-50882626ff1c service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:24.187601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf94b1c-dc13-4262-9344-50882626ff1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:24.222250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c101c75d12f452b99424cd38040ab40, user_id=fffdc9c626c14623950518dfc3e948d8, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:24.230928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf94b1c-dc13-4262-9344-50882626ff1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:24.232937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf94b1c-dc13-4262-9344-50882626ff1c service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:24.241195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:24.249866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 884/1751] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:24.294353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 868/1752] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:24 2022] PUT /identity/v3/projects/4c101c75d12f452b99424cd38040ab40/users/fffdc9c626c14623950518dfc3e948d8/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:24.310122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:24.310784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:24.310924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:24.408513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] MFA Rules not processed for user `fffdc9c626c14623950518dfc3e948d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:24.410309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:24.572586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 885/1753] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:24.577104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:24.757854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc542-b38a-4531-af7a-5595ae042faf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:24.815215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:24.815936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:24.816259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:24.816519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:24.817171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:24.823460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:24.901088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 869/1754] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:24 2022] POST /identity/v3/projects => generated 379 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:24.905513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:24.925838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79eafa-453f-4317-a8d8-ecb628ac2846 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:24.941264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:25.057080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:25.057691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.058000 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.058287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:25.058923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:25.060883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:25.105871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 886/1755] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:24 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:25.107480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:25.126598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5a4dc-e669-405c-9eea-42e67dbeeeae admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:25.132472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:25.134426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:25.141556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:25.142779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.143223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.143508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:25.144479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:25.146360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:25.160745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47a6a-1344-4c7f-8057-8bddf124c108 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:25.165581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:25.166235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.166515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.166784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] PATH_INFO: `/v3/users/f995a036c8b643fe9d182b925c34be97` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:25.170733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f995a036c8b643fe9d182b925c34be97)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:25.172714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:25.191137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 870/1756] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:25.200838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:25.251555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c554b5-acd5-4621-9ecd-4ab466d898f4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:25.263670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:25.264441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.264733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.265010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] PATH_INFO: `/v3/projects/c89fa49a76ac4d48a816e7bde34d39c4/users/d7dc7c569a6545e6bace9197575ff800/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:25.311875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f995a036c8b643fe9d182b925c34be97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:25.328616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f995a036c8b643fe9d182b925c34be97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:25.332346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89fa49a76ac4d48a816e7bde34d39c4, user_id=d7dc7c569a6545e6bace9197575ff800, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:25.339440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:25.377652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f995a036c8b643fe9d182b925c34be97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:25.378785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f995a036c8b643fe9d182b925c34be97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:25.385946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 871/1757] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:25 2022] PUT /identity/v3/projects/c89fa49a76ac4d48a816e7bde34d39c4/users/d7dc7c569a6545e6bace9197575ff800/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:25.390234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:25.394942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:25.403575 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f995a036c8b643fe9d182b925c34be97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:25.427259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 887/1758] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:25 2022] DELETE /identity/v3/users/f995a036c8b643fe9d182b925c34be97 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:25.431287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:25.663309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885189-b5e8-4160-9e13-bd0177838b24 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:25.791076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa9155-be60-474f-b8b8-6fe736877319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:25.791786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9155-be60-474f-b8b8-6fe736877319 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.792070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9155-be60-474f-b8b8-6fe736877319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.792418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9155-be60-474f-b8b8-6fe736877319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:25.793207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa9155-be60-474f-b8b8-6fe736877319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:25.795532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa9155-be60-474f-b8b8-6fe736877319 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:25.817933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 872/1759] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:25.819483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:25.841059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912cc4c-0160-4351-b04b-66015e834d28 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:25.848703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:25.850005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.850537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.851082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] PATH_INFO: `/v3/projects/c89fa49a76ac4d48a816e7bde34d39c4/users/d7dc7c569a6545e6bace9197575ff800/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:25.854205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89fa49a76ac4d48a816e7bde34d39c4, user_id=d7dc7c569a6545e6bace9197575ff800, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:25.858946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:25.872528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 888/1760] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:25 2022] PUT /identity/v3/projects/c89fa49a76ac4d48a816e7bde34d39c4/users/d7dc7c569a6545e6bace9197575ff800/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:25.883416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:25.883732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:25.884014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:26.019154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] MFA Rules not processed for user `d7dc7c569a6545e6bace9197575ff800`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:26.020607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:26.219001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 873/1761] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:26.230778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:26.232765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:26.366119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139c8f2-b380-450f-8479-4abeb4b7a127 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:26.383541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:26.384205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:26.384482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:26.384758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:26.405109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:26.407067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:26.420623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:26.443090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 889/1762] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:26.527005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:26.528790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:26.552068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc315c-a32c-45ed-bfb6-f6d2a5ee244e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:26.564179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:26.565151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:26.565576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:26.566062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:26.576347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:26.643793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:26.728927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:26.731775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:26.754130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 874/1763] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:27.943022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:27.944871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:28.127103 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629020a8-4660-43e7-8882-dfd28b54ad7b service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:28.135869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:28.137433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:28.138245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:28.139220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] PATH_INFO: `/v3/projects/5f3a7dac18ba421d9ff81fb4665419df` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:28.149059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3a7dac18ba421d9ff81fb4665419df)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:28.152499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:28.246467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3a7dac18ba421d9ff81fb4665419df', 'request_id': 'req-4dd55905-911f-4988-a7da-09935509d281'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.255171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3a7dac18ba421d9ff81fb4665419df', 'request_id': 'req-4dd55905-911f-4988-a7da-09935509d281'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.263495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3a7dac18ba421d9ff81fb4665419df', 'request_id': 'req-4dd55905-911f-4988-a7da-09935509d281'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.288590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] The token cache is being invalidate because project 5f3a7dac18ba421d9ff81fb4665419df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:28.289202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.290373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.295665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 890/1764] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:27 2022] DELETE /identity/v3/projects/5f3a7dac18ba421d9ff81fb4665419df => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:28.296592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:28.311195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b453e-e501-4d38-957b-e711961f069f service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:28.437212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:28.438183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:28.441546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:28.441995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] PATH_INFO: `/v3/users/d4e5c64295ca4c459571387891260895` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:28.443935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e5c64295ca4c459571387891260895)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:28.451265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:28.578258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e5c64295ca4c459571387891260895'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.603502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e5c64295ca4c459571387891260895'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.637654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e5c64295ca4c459571387891260895'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.640388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e5c64295ca4c459571387891260895'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.664736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1c2aa52f-0cfc-4975-8294-973a044fd328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e5c64295ca4c459571387891260895'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:28.688302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 875/1765] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:28 2022] DELETE /identity/v3/users/d4e5c64295ca4c459571387891260895 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:28.764902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:28.767540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:28.797936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd55905-911f-4988-a7da-09935509d281 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:29.049669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:29.052492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:29.052949 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:29.053580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] PATH_INFO: `/v3/users/e2601f50494749caa10565f0320a257f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:29.056920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2601f50494749caa10565f0320a257f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:29.063917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:29.146310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:29.176138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2601f50494749caa10565f0320a257f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:29.199484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2601f50494749caa10565f0320a257f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:29.244948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2601f50494749caa10565f0320a257f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:29.248615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2601f50494749caa10565f0320a257f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:29.274690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2601f50494749caa10565f0320a257f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:29.291416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 891/1766] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:28 2022] DELETE /identity/v3/users/e2601f50494749caa10565f0320a257f => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:29.294088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af92e5-262c-408f-8867-e2c97d5e3dc6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:29.294536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af92e5-262c-408f-8867-e2c97d5e3dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:29.295087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af92e5-262c-408f-8867-e2c97d5e3dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:29.383704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d9af92e5-262c-408f-8867-e2c97d5e3dc6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:29.386618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d9af92e5-262c-408f-8867-e2c97d5e3dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:29.486114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:29.517316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 876/1767] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:29.561486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:29.563755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:29.740909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7429-1229-4f3e-a3e2-7393666b0a1a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:29.747544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:29.749589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:29.750494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:29.751069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:29.762780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:29.764833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:29.778075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 892/1768] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:30.456450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:30.456923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:30.457020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:30.515038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] MFA Rules not processed for user `2b7e96d63d7446cfa3bc80784944c474`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:30.516494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:30.629021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:30.630724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:30.652690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2175-74ba-436c-a786-c29cab7f58b2 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:30.666270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:30.667122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:30.667404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:30.667741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] PATH_INFO: `/v3/projects/8031e2a2c56343c3981c76799d519147` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:30.669239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8031e2a2c56343c3981c76799d519147)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:30.674197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:30.712144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 877/1769] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:30.713259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:30.716304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:30.734292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f5d8-ba28-4ece-bc1e-dea91cf16184 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:30.740054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:30.741317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:30.741619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:30.741863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] PATH_INFO: `/v3/users/f5722a9f26114f33a97df455364f355e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:30.743574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5722a9f26114f33a97df455364f355e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:30.746713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:30.773129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8031e2a2c56343c3981c76799d519147', 'request_id': 'req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.780545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8031e2a2c56343c3981c76799d519147', 'request_id': 'req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.788643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8031e2a2c56343c3981c76799d519147', 'request_id': 'req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.805387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] The token cache is being invalidate because project 8031e2a2c56343c3981c76799d519147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:30.805731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.806494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.812542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5722a9f26114f33a97df455364f355e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.827650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 893/1770] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:30 2022] DELETE /identity/v3/projects/8031e2a2c56343c3981c76799d519147 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:30.829754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5722a9f26114f33a97df455364f355e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.832971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:30.834343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:30.850324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5722a9f26114f33a97df455364f355e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.851596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5722a9f26114f33a97df455364f355e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.871488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0fe259f6-6b63-485d-83be-f495ec5fb593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5722a9f26114f33a97df455364f355e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:30.882224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 878/1771] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:30 2022] DELETE /identity/v3/users/f5722a9f26114f33a97df455364f355e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:30.883970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe73d0-2968-42c3-80ee-e44e32acd782 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:30.884409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe73d0-2968-42c3-80ee-e44e32acd782 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:30.885026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe73d0-2968-42c3-80ee-e44e32acd782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:30.901119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:30.918993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0afe73d0-2968-42c3-80ee-e44e32acd782 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:30.920584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0afe73d0-2968-42c3-80ee-e44e32acd782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:30.936459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed64d9-c0f5-4996-a994-ebaf8ecae06d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.024366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.026804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.027609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.029063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:31.029564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.031218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.037198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.057117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 879/1772] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:31.058704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a2e8-aa2a-485d-9730-fa5527444183 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.059001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a2e8-aa2a-485d-9730-fa5527444183 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.059284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a2e8-aa2a-485d-9730-fa5527444183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.084248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c199a2e8-aa2a-485d-9730-fa5527444183 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:31.085964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c199a2e8-aa2a-485d-9730-fa5527444183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.096901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 880/1773] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:31.102805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.103104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.103387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.107712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 894/1774] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:30 2022] POST /identity/v3/projects => generated 371 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:31.112290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.115501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.139400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:31.141267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.233033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 881/1775] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:31.234266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.236346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.259932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4094c71-6c6c-4078-94f0-dd8a4542630d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.265543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.266347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.267685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.267965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.268616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.271186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.294589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 895/1776] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:31 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:31.295286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.301432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bab570-982f-41ed-b169-a22d656ac194 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.306233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.306854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.307138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.307468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.308323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe056b12-ce6f-40a0-be15-a4b9ec718414 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.314427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.315037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.315471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.315925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.316578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:31.317555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.319081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.319368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.321112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.332969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 882/1777] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:31.335563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 896/1778] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:31.338918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.342974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.351050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46c52-bcae-4699-93f7-e4cf75002e82 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.357600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.358267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.358710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.358963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.360654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.361809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.366387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd0ba4-35dd-41c6-971c-6b695cbe0bfb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.375430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.376874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.377172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.377436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.378125 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.380708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.384466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 883/1779] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:31.388872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.400394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ae474-98c5-4840-825d-c6c945bc1765 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.405590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.406394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.407350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.407350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] PATH_INFO: `/v3/projects/e8285ac516b44017820f6d41d4d08b2c/users/48f3c94371764ea4bf2418b351801e3f/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.428007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8285ac516b44017820f6d41d4d08b2c, user_id=48f3c94371764ea4bf2418b351801e3f, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.430959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 897/1780] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:31 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:31.432300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.435484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.455509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 884/1781] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:31 2022] PUT /identity/v3/projects/e8285ac516b44017820f6d41d4d08b2c/users/48f3c94371764ea4bf2418b351801e3f/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:31.456853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.473351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:31.537284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:31.543887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6481-0392-4c1e-8370-aa9e210908de admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.551048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.551690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.552043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.552385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.553732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.555584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.560752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa8f2e-f6bd-44b3-b621-fd8f7617b7bd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.568343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.569267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.570259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.570609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.571427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.573386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 885/1782] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:31.574863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.575751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.578081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.610718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1480118-dbf3-4b6f-9be4-5405cee58a6d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.612288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 898/1783] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:31 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:31.613757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.617224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.618283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.618633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.619000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] PATH_INFO: `/v3/projects/0fafaf0d34704d73b3cfd43bf4131dda` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.620262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fafaf0d34704d73b3cfd43bf4131dda)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.622630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.629897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71d586-ec23-4519-a17d-385d6e4942b5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.639093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.642302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.643277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.643875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] PATH_INFO: `/v3/projects/e8285ac516b44017820f6d41d4d08b2c/users/48f3c94371764ea4bf2418b351801e3f/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.646753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8285ac516b44017820f6d41d4d08b2c, user_id=48f3c94371764ea4bf2418b351801e3f, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.650488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.667447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 899/1784] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:31 2022] PUT /identity/v3/projects/e8285ac516b44017820f6d41d4d08b2c/users/48f3c94371764ea4bf2418b351801e3f/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:31.668354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.708319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fafaf0d34704d73b3cfd43bf4131dda', 'request_id': 'req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.715423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fafaf0d34704d73b3cfd43bf4131dda', 'request_id': 'req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.724684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fafaf0d34704d73b3cfd43bf4131dda', 'request_id': 'req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.743016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] The token cache is being invalidate because project 0fafaf0d34704d73b3cfd43bf4131dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:31.743314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.744103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.745632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 886/1785] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:31 2022] DELETE /identity/v3/projects/0fafaf0d34704d73b3cfd43bf4131dda => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:31.749675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.751778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.796142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5ee99-d411-4664-9b7a-7709525a13f2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.833124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdfe89-fb34-4ca9-9bdd-b49e5cda1f16 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.838364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.839095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.839554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.839900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] PATH_INFO: `/v3/users/a7b4d6e47dca438084fa880f3f2f8293` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.841170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b4d6e47dca438084fa880f3f2f8293)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.843566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.861237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.861873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.862189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.862462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.863371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.865016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.880168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 900/1786] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:31.881233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.883548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.899793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58671ab2-3092-43bb-83a9-61bd60713e50 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.904254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b4d6e47dca438084fa880f3f2f8293'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.905814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.906694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.906979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.908234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.908892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.911006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:31.913147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b4d6e47dca438084fa880f3f2f8293'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.927123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 901/1787] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:31.936169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.939050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b4d6e47dca438084fa880f3f2f8293'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.940036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b4d6e47dca438084fa880f3f2f8293'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.946579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:31.947862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7803-0a5c-46df-b7ce-5a4b508d3024 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:31.950624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b4d6e47dca438084fa880f3f2f8293'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:31.953817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c5a7-7d3f-4afb-8724-a0adb929f3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:31.954608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c5a7-7d3f-4afb-8724-a0adb929f3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:31.954903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c5a7-7d3f-4afb-8724-a0adb929f3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:31.955223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c5a7-7d3f-4afb-8724-a0adb929f3f2 admin admin] PATH_INFO: `/v3/projects/cec8e339d755416fbdc67477e6022dc5/users/8ae2be3e5db640859f5a4fc8f9229756/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:31.960346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 887/1788] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:31 2022] DELETE /identity/v3/users/a7b4d6e47dca438084fa880f3f2f8293 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:31.961817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:31.964980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:31.976832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c5a7-7d3f-4afb-8724-a0adb929f3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec8e339d755416fbdc67477e6022dc5, user_id=8ae2be3e5db640859f5a4fc8f9229756, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:31.979582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c5a7-7d3f-4afb-8724-a0adb929f3f2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:32.001959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 902/1789] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:31 2022] PUT /identity/v3/projects/cec8e339d755416fbdc67477e6022dc5/users/8ae2be3e5db640859f5a4fc8f9229756/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:32.011963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:32.012269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:32.012827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:32.027755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:32.048824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] MFA Rules not processed for user `8ae2be3e5db640859f5a4fc8f9229756`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:32.050418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:32.075287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf5f86-49db-41dd-a0bd-27ea1b5bf083 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:32.162716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 903/1790] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:32.169439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:32.170691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:32.198492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:32.199260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:32.199548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:32.199826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] PATH_INFO: `/v3/projects/e492221e865242078e186f68f9ebd96f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:32.201117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e492221e865242078e186f68f9ebd96f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:32.203060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:32.277138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e492221e865242078e186f68f9ebd96f', 'request_id': 'req-5a396546-f8e2-424a-96b8-83b63ee13cfe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.283099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afc9e8-d6dd-4514-9af0-ec08d9f85ab4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:32.285732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e492221e865242078e186f68f9ebd96f', 'request_id': 'req-5a396546-f8e2-424a-96b8-83b63ee13cfe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.296698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e492221e865242078e186f68f9ebd96f', 'request_id': 'req-5a396546-f8e2-424a-96b8-83b63ee13cfe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.334714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] The token cache is being invalidate because project e492221e865242078e186f68f9ebd96f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:32.336501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.340083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.343975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 888/1791] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:31 2022] DELETE /identity/v3/projects/e492221e865242078e186f68f9ebd96f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:32.349510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:32.359355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:32.453222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:32.462320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:32.463017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:32.463240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:32.463606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:32.464454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:32.466430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:32.519557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a396546-f8e2-424a-96b8-83b63ee13cfe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:32.525974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 904/1792] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:32 2022] POST /identity/v3/projects => generated 381 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:32.531027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:32.534235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:32.537574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:32.538908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:32.539211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:32.539498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] PATH_INFO: `/v3/users/fffdc9c626c14623950518dfc3e948d8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:32.541289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffdc9c626c14623950518dfc3e948d8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:32.569554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:32.639498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:32.699250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffdc9c626c14623950518dfc3e948d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.721465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffdc9c626c14623950518dfc3e948d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.730518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d82812-598c-4606-97f2-b813fb0f913d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:32.740296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:32.742152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:32.743306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:32.743870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:32.745374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:32.748254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:32.748429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffdc9c626c14623950518dfc3e948d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.752132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffdc9c626c14623950518dfc3e948d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.767473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffdc9c626c14623950518dfc3e948d8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:32.777278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 889/1793] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:32 2022] DELETE /identity/v3/users/fffdc9c626c14623950518dfc3e948d8 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:32.790961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:32.799263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:32.810027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 905/1794] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:32 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:32.813107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:32.822708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:32.860244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e20645-6e0d-4cbf-8899-f3ab69261f2d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.031768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1add8-5806-42ed-9713-0edbc013259b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.042494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.042494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.042494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.042494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] PATH_INFO: `/v3/projects/2c6ad4f55a414db9a509be49690612b9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.042922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6ad4f55a414db9a509be49690612b9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.043323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.046446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.047328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.048361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.049199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.052238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.056597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.090587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 906/1795] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:33.095309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.111298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba8c58-0b31-4eb6-a9f1-fadfbe93248e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.137190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6ad4f55a414db9a509be49690612b9', 'request_id': 'req-8cf7e001-bf61-42d1-9306-082f89346632'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.140584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:33.154036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6ad4f55a414db9a509be49690612b9', 'request_id': 'req-8cf7e001-bf61-42d1-9306-082f89346632'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.163302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6ad4f55a414db9a509be49690612b9', 'request_id': 'req-8cf7e001-bf61-42d1-9306-082f89346632'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.188215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] The token cache is being invalidate because project 2c6ad4f55a414db9a509be49690612b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:33.189149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.189926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8cf7e001-bf61-42d1-9306-082f89346632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.198386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 890/1796] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:32 2022] DELETE /identity/v3/projects/2c6ad4f55a414db9a509be49690612b9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:33.200171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d25a-7195-4c6a-b8b1-1e005391b571 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.200474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d25a-7195-4c6a-b8b1-1e005391b571 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.200764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d25a-7195-4c6a-b8b1-1e005391b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.240036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7266d25a-7195-4c6a-b8b1-1e005391b571 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:33.240939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7266d25a-7195-4c6a-b8b1-1e005391b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.264258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.267787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.268350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.270204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] PATH_INFO: `/v3/projects/e89112095a8b4d499bbeb585c7f52ac8/users/e6224cdb4a504f5ba3c1b5773c4d3ab9/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.302727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89112095a8b4d499bbeb585c7f52ac8, user_id=e6224cdb4a504f5ba3c1b5773c4d3ab9, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.305931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.338760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 907/1797] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:33 2022] PUT /identity/v3/projects/e89112095a8b4d499bbeb585c7f52ac8/users/e6224cdb4a504f5ba3c1b5773c4d3ab9/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:33.339853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.347087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.392180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 891/1798] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:33.400431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72af0d-e249-414b-aecf-215638b92347 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.402192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72af0d-e249-414b-aecf-215638b92347 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.402688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72af0d-e249-414b-aecf-215638b92347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.433111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5d72af0d-e249-414b-aecf-215638b92347 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:33.434532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5d72af0d-e249-414b-aecf-215638b92347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.458567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:33.500914 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b91ac-0e73-46d2-9c4d-17b28000f860 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.506738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.507566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.507861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.508137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.508787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.510557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.520196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 892/1799] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:33.520196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.520196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.520196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.525860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 908/1800] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:33.526846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.530171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.564084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2996f-63bf-40b2-99f0-d2d758221cd0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.570552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.572006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.572290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.572566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] PATH_INFO: `/v3/projects/4c101c75d12f452b99424cd38040ab40` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.574122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c101c75d12f452b99424cd38040ab40)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.576118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.587812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] MFA Rules not processed for user `48f3c94371764ea4bf2418b351801e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:33.589114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.684156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 893/1801] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:33.684351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c101c75d12f452b99424cd38040ab40', 'request_id': 'req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.684771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.692120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c101c75d12f452b99424cd38040ab40', 'request_id': 'req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.695374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6922e3e-b24c-4ad2-b304-b6e6b77316d5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.699947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:33.710586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c101c75d12f452b99424cd38040ab40', 'request_id': 'req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.746668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] The token cache is being invalidate because project 4c101c75d12f452b99424cd38040ab40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:33.746965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.747665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:33.749550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 909/1802] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:33 2022] DELETE /identity/v3/projects/4c101c75d12f452b99424cd38040ab40 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:33.750378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.756664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.833967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.834892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.835384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.835728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.836451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:33.836824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.840340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.845591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e3fdb-0bcc-46e4-b369-740c7c57f5cb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:33.855151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:33.860532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:33.860985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:33.861427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] PATH_INFO: `/v3/projects/e89112095a8b4d499bbeb585c7f52ac8/users/e6224cdb4a504f5ba3c1b5773c4d3ab9/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:33.865147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89112095a8b4d499bbeb585c7f52ac8, user_id=e6224cdb4a504f5ba3c1b5773c4d3ab9, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:33.869165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:33.874948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 894/1803] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:33.877224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.883957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:33.884509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 910/1804] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:33 2022] PUT /identity/v3/projects/e89112095a8b4d499bbeb585c7f52ac8/users/e6224cdb4a504f5ba3c1b5773c4d3ab9/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:33.885523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:33.896047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:34.010893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:34.033834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc71c0-4354-4c44-965a-5a52152fff3b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.043387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e7be-864b-4207-98d2-c080713c5d7d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.052764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.053415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.053718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.054022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] PATH_INFO: `/v3/users/d7dc7c569a6545e6bace9197575ff800` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.056634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7dc7c569a6545e6bace9197575ff800)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.058063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.112098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.112940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.113692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.114116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.116385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:34.135726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7dc7c569a6545e6bace9197575ff800'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:34.150561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7dc7c569a6545e6bace9197575ff800'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:34.172173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7dc7c569a6545e6bace9197575ff800'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:34.172988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7dc7c569a6545e6bace9197575ff800'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:34.184311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7dc7c569a6545e6bace9197575ff800'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:34.196794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 911/1805] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:33 2022] DELETE /identity/v3/users/d7dc7c569a6545e6bace9197575ff800 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:34.196794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:34.198301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:34.199110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.200849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45945b-f864-4181-82db-cf604de7f7c5 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.214282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 895/1806] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:34.231555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.231855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.232136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.283876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:34.296444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] MFA Rules not processed for user `e6224cdb4a504f5ba3c1b5773c4d3ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:34.298015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:34.299254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e58cc3-2e0e-4fa2-9903-56b7bcfcdfc3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.311401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.312555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.313126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.314176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.315607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.318721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.369568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 912/1807] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:34 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:34.373955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:34.429808 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 896/1808] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:34.452147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:34.453792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:34.463803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:34.472548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177abd4e-d0fd-4eaf-8004-7a4f24424895 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.480668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.481381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.481727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.482127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.483132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.485860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.517019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 913/1809] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:34 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:34.521701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:34.551333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76182690-c675-4ab8-bead-d45c4c95d07b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.583443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.583443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.583443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.583443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.584059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.584059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.614711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18979b-8a70-4fae-b50f-2524698f1e91 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.634440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 914/1810] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:34.639922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:34.664467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c059b1-2a7a-4ddc-8f1f-c16d4be5419c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.689885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.691958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.694263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.695835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.698147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.704929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.742506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 915/1811] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:34.745549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:34.763253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e32328-b987-4321-a6a1-d0efe69c375b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:34.768369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.769002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.769286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.769564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.777302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06fc4c-fb64-4574-95c2-147ff626020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:34.778018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06fc4c-fb64-4574-95c2-147ff626020b admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.778301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06fc4c-fb64-4574-95c2-147ff626020b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.778578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06fc4c-fb64-4574-95c2-147ff626020b admin admin] PATH_INFO: `/v3/projects/6d21cffb715f4d6189f12dd70a75d92c/users/22cc543135384935bb2bbbd863c4ff7e/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.822721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06fc4c-fb64-4574-95c2-147ff626020b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d21cffb715f4d6189f12dd70a75d92c, user_id=22cc543135384935bb2bbbd863c4ff7e, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.822721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06fc4c-fb64-4574-95c2-147ff626020b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.864665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 916/1812] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:34 2022] PUT /identity/v3/projects/6d21cffb715f4d6189f12dd70a75d92c/users/22cc543135384935bb2bbbd863c4ff7e/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:34.867489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:34.867489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:34.867489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:34.936111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] MFA Rules not processed for user `22cc543135384935bb2bbbd863c4ff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:34.938062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:34.939077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:34.939745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:34.952152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 897/1813] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:35.037669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 917/1814] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:35.042887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.049812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:35.064789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e94c4-93b7-49f8-b58d-04b0f5c73f47 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.104560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.160885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.161812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.162296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.162741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.163636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.165856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.213850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 898/1815] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:35 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:35.217576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.248174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa25dca-1360-4a89-9002-9283be32b765 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.361843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:35.404627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624007d4-70a3-4cfd-991e-721a85eac61c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.405682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.406522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.407086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.407520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.411237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:35.412041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca718af-3017-4981-a410-a0e5e6678e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.413891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca718af-3017-4981-a410-a0e5e6678e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.414904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca718af-3017-4981-a410-a0e5e6678e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.417972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca718af-3017-4981-a410-a0e5e6678e88 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.418770 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca718af-3017-4981-a410-a0e5e6678e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.421096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca718af-3017-4981-a410-a0e5e6678e88 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.463132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 899/1816] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:35 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:35.464493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce8fb6-c4d1-4f05-a30f-99010764e6f7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.464861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce8fb6-c4d1-4f05-a30f-99010764e6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.465325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce8fb6-c4d1-4f05-a30f-99010764e6f7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.471633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 900/1817] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:00:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:00:35.472595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce8fb6-c4d1-4f05-a30f-99010764e6f7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.506706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce8fb6-c4d1-4f05-a30f-99010764e6f7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.510939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.512634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95382072-e3bb-4577-924d-3234abfa8562 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.516877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.517508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.517793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.518666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.519335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.521218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.553609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 918/1819] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:35.553731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 901/1819] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:35.554175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.554817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.555135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.555866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.580812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c5694-d2dd-4306-b5b8-d887d2327557 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.593839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.595067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.595357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.595634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] PATH_INFO: `/v3/projects/74e5065fa44e4c37b6b66e0006d2b9d7/users/b1de4aafe1964fa1879f46505fe75637/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.607648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:35.611136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:35.635592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e5065fa44e4c37b6b66e0006d2b9d7, user_id=b1de4aafe1964fa1879f46505fe75637, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.643566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.675835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 902/1820] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:35 2022] PUT /identity/v3/projects/74e5065fa44e4c37b6b66e0006d2b9d7/users/b1de4aafe1964fa1879f46505fe75637/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:35.680425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.754642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:35.791093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22d6f8-7f5b-4cc1-bd87-f6b2ac8405ab admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.798804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.799763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.800096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.801126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.801971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.805607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.824735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 903/1821] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:35.826858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 919/1822] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:00:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:35.845762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.861270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.881197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a20c-468f-4c53-a80a-f591ed6ce9da admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.890633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:35.906550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:35.907865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:35.908087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:35.908367 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] PATH_INFO: `/v3/projects/74e5065fa44e4c37b6b66e0006d2b9d7/users/b1de4aafe1964fa1879f46505fe75637/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:35.913106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e5065fa44e4c37b6b66e0006d2b9d7, user_id=b1de4aafe1964fa1879f46505fe75637, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:35.921939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:35.922097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d8fb6-e9fc-40d7-afa4-820a8aa97681 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:35.944269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 904/1823] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:35 2022] PUT /identity/v3/projects/74e5065fa44e4c37b6b66e0006d2b9d7/users/b1de4aafe1964fa1879f46505fe75637/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:35.956280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:35.958789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.010201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bee59-315f-4d8b-8019-8fb5db0403e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.010818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bee59-315f-4d8b-8019-8fb5db0403e3 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.011253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bee59-315f-4d8b-8019-8fb5db0403e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.011565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bee59-315f-4d8b-8019-8fb5db0403e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.022838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bee59-315f-4d8b-8019-8fb5db0403e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.024610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bee59-315f-4d8b-8019-8fb5db0403e3 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.026419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676310e-cf65-40d0-aceb-ea444f863c35 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.031496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.032730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.033017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.033299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] PATH_INFO: `/v3/projects/c89fa49a76ac4d48a816e7bde34d39c4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.034425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89fa49a76ac4d48a816e7bde34d39c4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.035746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 920/1824] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:36.036427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.036944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32de47-de29-4ab4-b6b2-3b4da83421c4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.037232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32de47-de29-4ab4-b6b2-3b4da83421c4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.037505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32de47-de29-4ab4-b6b2-3b4da83421c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.069013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5e32de47-de29-4ab4-b6b2-3b4da83421c4 None None] MFA Rules not processed for user `b1de4aafe1964fa1879f46505fe75637`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:36.070327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5e32de47-de29-4ab4-b6b2-3b4da83421c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.079487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:36.096699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89fa49a76ac4d48a816e7bde34d39c4', 'request_id': 'req-7c37c43e-0343-4985-8997-06d60df94a25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.102511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89fa49a76ac4d48a816e7bde34d39c4', 'request_id': 'req-7c37c43e-0343-4985-8997-06d60df94a25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.109486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89fa49a76ac4d48a816e7bde34d39c4', 'request_id': 'req-7c37c43e-0343-4985-8997-06d60df94a25'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.124667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] The token cache is being invalidate because project c89fa49a76ac4d48a816e7bde34d39c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:36.125019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.125623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.131484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 905/1825] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:35 2022] DELETE /identity/v3/projects/c89fa49a76ac4d48a816e7bde34d39c4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:36.131484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.131484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.167122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 921/1826] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:36.168803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.169092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.169704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.195410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:36.200222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.225684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37c43e-0343-4985-8997-06d60df94a25 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.232595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.233381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.233790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.234177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.236396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.264159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:36.332496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 922/1827] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:36.338204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.338204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.366135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bf88a-6a07-444d-9843-9d1eaf12bf07 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.370659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.371292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.371579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.371860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.381739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.384683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34beb841-20d9-4871-b6b6-8d3af3f13391 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.398626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 906/1828] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:36.405954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.406251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.406532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.430778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:36.432056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.450461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 907/1829] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:36.455134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.465747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.467334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.470448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1de6b4-ca7f-43e7-b42a-1298f0044604 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.475867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.476713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.477200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.477704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.478867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:36.479524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.480924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 923/1830] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:36.483778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.507604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 908/1831] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:36.513067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.520442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.523299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.529328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935cf4-af23-4fb4-b66a-972e7e92d3f1 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.535414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.536998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.537465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.537896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.539190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.539314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b26ef-7e91-47e3-b6c8-5acde7d0b15c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.541323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.545375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.546039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.546374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.546710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] PATH_INFO: `/v3/users/22cc543135384935bb2bbbd863c4ff7e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.548832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=22cc543135384935bb2bbbd863c4ff7e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.551382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.589585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 924/1832] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:36 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:36.590820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.609847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:36.665526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22cc543135384935bb2bbbd863c4ff7e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.677248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22cc543135384935bb2bbbd863c4ff7e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.714319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22cc543135384935bb2bbbd863c4ff7e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.716080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22cc543135384935bb2bbbd863c4ff7e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.740303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56969ebf-a1f9-4f5b-b693-ac09b6a36ee1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.747245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84558241-6523-49c3-887a-9c998861ea5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22cc543135384935bb2bbbd863c4ff7e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:36.753482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.754342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.754466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.754657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.755422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.757396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.762588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 909/1833] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:36 2022] DELETE /identity/v3/users/22cc543135384935bb2bbbd863c4ff7e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:36.767583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97411-ba10-46af-a056-f457dbad09be None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.767903 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97411-ba10-46af-a056-f457dbad09be None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.768192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97411-ba10-46af-a056-f457dbad09be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.790203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-43e97411-ba10-46af-a056-f457dbad09be None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:36.793708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43e97411-ba10-46af-a056-f457dbad09be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.794449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 925/1834] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:36 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:36.796466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.798364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.813254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03f3b6-8700-4632-bc23-014fb18f3d2c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:36.870219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:36.897953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 910/1835] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:36.909790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e97411-ba10-46af-a056-f457dbad09be None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:36.911990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43e97411-ba10-46af-a056-f457dbad09be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:36.927540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:36.929127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:36.929583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:36.930079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:36.931004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:36.934703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:36.967869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 926/1836] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:36.972222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.004092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a7777-5939-4a31-9179-1cd04df08fa5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.009713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.010383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.010671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.010947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.011606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.013330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.027385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 927/1837] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:37.035978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e97411-ba10-46af-a056-f457dbad09be None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.040853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b99426-fab6-4615-b9a5-05f71bf76f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.041677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b99426-fab6-4615-b9a5-05f71bf76f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.042055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b99426-fab6-4615-b9a5-05f71bf76f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.042779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b99426-fab6-4615-b9a5-05f71bf76f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.045426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.059181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b99426-fab6-4615-b9a5-05f71bf76f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.061212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b99426-fab6-4615-b9a5-05f71bf76f0b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.071469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5781c0-383e-44ab-baa8-8efded2ad9c9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.078871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 911/1838] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:37.084043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.084737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.085020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.085296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] PATH_INFO: `/v3/projects/f91cf1a9640b4ff08c7d6f4c9f5f48ee/users/cf3a52d7b5384ecc9e46b4a2dacd252d/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.118480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91cf1a9640b4ff08c7d6f4c9f5f48ee, user_id=cf3a52d7b5384ecc9e46b4a2dacd252d, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.121819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.151171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 928/1839] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:37 2022] PUT /identity/v3/projects/f91cf1a9640b4ff08c7d6f4c9f5f48ee/users/cf3a52d7b5384ecc9e46b4a2dacd252d/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:37.163959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.164255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.164537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.207853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] MFA Rules not processed for user `cf3a52d7b5384ecc9e46b4a2dacd252d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:37.209766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:37.362887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 912/1840] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:37.378424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.481527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8337689-0a07-4dd7-a705-cb16ea504044 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.491265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.492082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.492527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.492855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.494185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.497564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.523933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:37.550443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 929/1841] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:37 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:37.556961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.576800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dce8f-0f16-4584-8f9a-4e09525e59d1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.639742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.643913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:37.667678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.668363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.668653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.668933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.670102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.672484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.701119 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 913/1842] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:37 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:37.705085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.717487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b3b2e-7acd-441d-aa6a-37a84fbc7b46 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.721481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bedda-8a59-4d5a-852a-5395b759f4f1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.722396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.723067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.723296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.723576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.724238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.725842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.728426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.729061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.729338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.729798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] PATH_INFO: `/v3/projects/6d21cffb715f4d6189f12dd70a75d92c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.730961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d21cffb715f4d6189f12dd70a75d92c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.733685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.743349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 914/1843] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:37.744290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.758810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c07c7-c6d6-41ca-9104-3957a5e2d078 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:37.766256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:37.767154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:37.770391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:37.770873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] PATH_INFO: `/v3/projects/2692e18eca0447739862c20f5527ed7d/users/eb13dad6ff7b45e5ae9a3a585780cb1b/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:37.775780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:37.827952 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2692e18eca0447739862c20f5527ed7d, user_id=eb13dad6ff7b45e5ae9a3a585780cb1b, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:37.831684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:37.857539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 915/1844] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:37 2022] PUT /identity/v3/projects/2692e18eca0447739862c20f5527ed7d/users/eb13dad6ff7b45e5ae9a3a585780cb1b/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:37.864690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.867390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d21cffb715f4d6189f12dd70a75d92c', 'request_id': 'req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:37.890178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d21cffb715f4d6189f12dd70a75d92c', 'request_id': 'req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:37.906560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d21cffb715f4d6189f12dd70a75d92c', 'request_id': 'req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:37.932600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] The token cache is being invalidate because project 6d21cffb715f4d6189f12dd70a75d92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:37.934658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:37.935801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:37.940177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 930/1845] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:37 2022] DELETE /identity/v3/projects/6d21cffb715f4d6189f12dd70a75d92c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:37.944576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:37.946652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:37.996065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea3131-f54c-40c3-984f-6ca9209a2b13 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:38.005380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:38.008819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.009383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.009916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:38.012513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:38.019264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:38.045348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 916/1846] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:38.059248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:38.061098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:38.078797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef14b-c504-458c-9fc7-ff49d1d1a9df admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:38.091017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd7ea7-5bec-40a7-9178-90ecdd6fade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:38.091720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd7ea7-5bec-40a7-9178-90ecdd6fade1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.092009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd7ea7-5bec-40a7-9178-90ecdd6fade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.092290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd7ea7-5bec-40a7-9178-90ecdd6fade1 admin admin] PATH_INFO: `/v3/projects/2692e18eca0447739862c20f5527ed7d/users/eb13dad6ff7b45e5ae9a3a585780cb1b/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:38.097546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd7ea7-5bec-40a7-9178-90ecdd6fade1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2692e18eca0447739862c20f5527ed7d, user_id=eb13dad6ff7b45e5ae9a3a585780cb1b, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:38.106274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1eabfa-5050-4451-a0ec-fa5d90dac9d4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:38.112293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:38.113296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.113640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.114096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] PATH_INFO: `/v3/users/b1de4aafe1964fa1879f46505fe75637` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:38.117067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1de4aafe1964fa1879f46505fe75637)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:38.117233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd7ea7-5bec-40a7-9178-90ecdd6fade1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:38.118993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:38.142893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 917/1847] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:38 2022] PUT /identity/v3/projects/2692e18eca0447739862c20f5527ed7d/users/eb13dad6ff7b45e5ae9a3a585780cb1b/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:38.158117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.158436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.160391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:38.167093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:38.218075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1de4aafe1964fa1879f46505fe75637'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.236336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] MFA Rules not processed for user `eb13dad6ff7b45e5ae9a3a585780cb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:38.237348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1de4aafe1964fa1879f46505fe75637'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.240141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:38.253720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1de4aafe1964fa1879f46505fe75637'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.255054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1de4aafe1964fa1879f46505fe75637'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.271327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1de4aafe1964fa1879f46505fe75637'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.289143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 931/1848] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:37 2022] DELETE /identity/v3/users/b1de4aafe1964fa1879f46505fe75637 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:38.383891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 918/1849] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:38.399361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:38.404048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:38.493108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00630949-da65-4bb8-8dbb-dda20f81d071 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:38.498829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117405-3588-42ac-86ff-ff2a6435d8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:38.499619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117405-3588-42ac-86ff-ff2a6435d8da service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.500060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117405-3588-42ac-86ff-ff2a6435d8da service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.500338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117405-3588-42ac-86ff-ff2a6435d8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:38.571630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117405-3588-42ac-86ff-ff2a6435d8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:38.573101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117405-3588-42ac-86ff-ff2a6435d8da service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:38.585257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 932/1850] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:38.668100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:38.670835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:38.739457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:38.757508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67408178-7190-4c49-ad25-5ee2810124f1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:38.825151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:38.825982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.826792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.827153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] PATH_INFO: `/v3/projects/74e5065fa44e4c37b6b66e0006d2b9d7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:38.829657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e5065fa44e4c37b6b66e0006d2b9d7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:38.832943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:38.897031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e5065fa44e4c37b6b66e0006d2b9d7', 'request_id': 'req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.905167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e5065fa44e4c37b6b66e0006d2b9d7', 'request_id': 'req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.914620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e5065fa44e4c37b6b66e0006d2b9d7', 'request_id': 'req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.946757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] The token cache is being invalidate because project 74e5065fa44e4c37b6b66e0006d2b9d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:38.947058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.947760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8b8ef9b4-e01d-4386-93d5-6af38ff3ec2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:38.949692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 919/1851] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:38 2022] DELETE /identity/v3/projects/74e5065fa44e4c37b6b66e0006d2b9d7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:38.956715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:38.957162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:38.957965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.003444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:39.010953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.146093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:39.169354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 933/1852] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:39.177385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.177677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.178334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.208692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.211168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.212224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:39.213417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.228717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 920/1853] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:39.232703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.237224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09268c7f-02fd-42de-8829-3a3d556e369f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.248889 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.251921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.252521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.253010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] PATH_INFO: `/v3/users/cf3a52d7b5384ecc9e46b4a2dacd252d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.254888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3a52d7b5384ecc9e46b4a2dacd252d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.257535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.260825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e38e2e-52b1-4752-acfe-40be9596e8c9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.278811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.281476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.282051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.282422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.284857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:39.285443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.291778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.333304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 921/1854] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:00:39.336042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3a52d7b5384ecc9e46b4a2dacd252d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:39.337018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.347098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3a52d7b5384ecc9e46b4a2dacd252d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:39.361271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ea48-8c17-4732-9f65-8c6d86ec655f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.375012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3a52d7b5384ecc9e46b4a2dacd252d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:39.377321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3a52d7b5384ecc9e46b4a2dacd252d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:39.386796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.388387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.388963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.390058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.391308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.394330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.405984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8f1b3ed0-2e18-4d9b-9da6-992d95a411b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3a52d7b5384ecc9e46b4a2dacd252d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:39.423858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 934/1855] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:39 2022] DELETE /identity/v3/users/cf3a52d7b5384ecc9e46b4a2dacd252d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:39.428639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.428939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.429217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.476579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:39.478100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.482990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 922/1856] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:39 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:39.483934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.485616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.549650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:39.567446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 935/1857] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:39.570706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e91ab-fc41-4cab-a0d0-785163b3bf03 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.577745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.578675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.579525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.579802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.580374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.580474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.582539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.583068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.635582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 923/1858] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:39 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:39.641071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.663434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6323e-2a3a-4cd7-a6f6-902607bef00a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.671929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.673873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.675659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.676085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.677241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b850361-0b81-4357-a4e1-991be7096cf9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.678101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.679905 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.689942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.697157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.697432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.697702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.710491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 924/1859] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:39.711403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.718074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.719553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.727780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e36ee-16e8-4088-a69f-3b734d510644 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.734156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 936/1860] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:39.737867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fca3b-c531-49c2-b1cb-6edfcc71939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.739103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fca3b-c531-49c2-b1cb-6edfcc71939f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.739693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fca3b-c531-49c2-b1cb-6edfcc71939f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.740055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fca3b-c531-49c2-b1cb-6edfcc71939f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.740849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fca3b-c531-49c2-b1cb-6edfcc71939f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.742680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fca3b-c531-49c2-b1cb-6edfcc71939f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.755878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 925/1861] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:39.760080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:39.767088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:39.771298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da945cde-1dfe-4589-9df4-74a6f7882000 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:39.779014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:39.779935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.780658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.781430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] PATH_INFO: `/v3/projects/0375f4b01b1a4a90acbb5a79c4537a05/users/088135830d88482390d35745733dfa15/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.799066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0375f4b01b1a4a90acbb5a79c4537a05, user_id=088135830d88482390d35745733dfa15, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:39.801802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:39.817451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 937/1862] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:39 2022] PUT /identity/v3/projects/0375f4b01b1a4a90acbb5a79c4537a05/users/088135830d88482390d35745733dfa15/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:39.827782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:39.828196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:39.828630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:39.861341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] MFA Rules not processed for user `088135830d88482390d35745733dfa15`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:39.862990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:39.936242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 926/1863] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:39.941485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.018173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2545d1-6b64-4f23-8ef1-fbd510c7c5d6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.024767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.025391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.025671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.025974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.026650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.029419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.065272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 938/1864] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:39 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:40.069189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.080884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c67723-1d6a-4471-ab93-4b5bcb637b09 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.095404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.097682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:40.163338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.163946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.164240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.164499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.165142 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.167070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.182770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c1a0d-41ca-4f09-87a0-aff7aee7681a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.188875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.189625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.190055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.190395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] PATH_INFO: `/v3/projects/f91cf1a9640b4ff08c7d6f4c9f5f48ee` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.191711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91cf1a9640b4ff08c7d6f4c9f5f48ee)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.193245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 927/1865] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:40 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:40.194808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.198572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.208388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d95ac-6322-4e44-af79-8a951fa22588 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.213059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.213653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.213980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.214261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.214936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.216512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.233507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 928/1866] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:40.234667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.248683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa6c4c-2ed6-4536-97d8-6d29aefc85f0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.259765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91cf1a9640b4ff08c7d6f4c9f5f48ee', 'request_id': 'req-667c3a7a-de1e-40fd-a410-511f4c0b8861'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.267072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91cf1a9640b4ff08c7d6f4c9f5f48ee', 'request_id': 'req-667c3a7a-de1e-40fd-a410-511f4c0b8861'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.275437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:40.275648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91cf1a9640b4ff08c7d6f4c9f5f48ee', 'request_id': 'req-667c3a7a-de1e-40fd-a410-511f4c0b8861'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.279717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:40.294483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] The token cache is being invalidate because project f91cf1a9640b4ff08c7d6f4c9f5f48ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:40.294818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.295891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.298234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 939/1867] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:40 2022] DELETE /identity/v3/projects/f91cf1a9640b4ff08c7d6f4c9f5f48ee => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:40.302324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.303744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:40.318653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.319242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.319527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.319807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] PATH_INFO: `/v3/projects/99e4b264441445edb4a35b21d2b2c9f2/users/6ffb8dbf5c7c411085cc42b2acd06596/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.334379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e4b264441445edb4a35b21d2b2c9f2, user_id=6ffb8dbf5c7c411085cc42b2acd06596, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.337047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.351130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 929/1868] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:40 2022] PUT /identity/v3/projects/99e4b264441445edb4a35b21d2b2c9f2/users/6ffb8dbf5c7c411085cc42b2acd06596/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:40.355318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.356948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:40.375487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c3a7a-de1e-40fd-a410-511f4c0b8861 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.431334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456229ce-c546-42d0-be6b-3b22f8f0b417 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.436064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.436703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.436925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.437043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.437250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.437921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.438051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.438581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.439083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] PATH_INFO: `/v3/users/eb13dad6ff7b45e5ae9a3a585780cb1b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.439579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.441052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb13dad6ff7b45e5ae9a3a585780cb1b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.443975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.452890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 930/1869] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:40.454594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.466139 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88483375-126b-4613-8e71-34b69acad0e8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.472282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba2711-2930-40fd-a0d6-4b6392891762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.473062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2711-2930-40fd-a0d6-4b6392891762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.473355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2711-2930-40fd-a0d6-4b6392891762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.474188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2711-2930-40fd-a0d6-4b6392891762 admin admin] PATH_INFO: `/v3/projects/99e4b264441445edb4a35b21d2b2c9f2/users/6ffb8dbf5c7c411085cc42b2acd06596/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.475878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba2711-2930-40fd-a0d6-4b6392891762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e4b264441445edb4a35b21d2b2c9f2, user_id=6ffb8dbf5c7c411085cc42b2acd06596, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.479214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba2711-2930-40fd-a0d6-4b6392891762 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.496538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 931/1870] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:40 2022] PUT /identity/v3/projects/99e4b264441445edb4a35b21d2b2c9f2/users/6ffb8dbf5c7c411085cc42b2acd06596/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:40.504145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb13dad6ff7b45e5ae9a3a585780cb1b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.512163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.512553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.512879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.516727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb13dad6ff7b45e5ae9a3a585780cb1b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.534922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb13dad6ff7b45e5ae9a3a585780cb1b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.543050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb13dad6ff7b45e5ae9a3a585780cb1b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.555025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb13dad6ff7b45e5ae9a3a585780cb1b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:40.555476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] MFA Rules not processed for user `6ffb8dbf5c7c411085cc42b2acd06596`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:40.559936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:40.567261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 940/1871] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:40 2022] DELETE /identity/v3/users/eb13dad6ff7b45e5ae9a3a585780cb1b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:40.652606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 932/1872] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:40.664096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:40.665852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:40.695865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:40.750474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa4da8-ae3d-4f8a-a955-0623c1c6299b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:40.756160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b3e3-a2e8-4dc6-b64f-35731b8d5466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:40.756780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b3e3-a2e8-4dc6-b64f-35731b8d5466 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:40.757061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b3e3-a2e8-4dc6-b64f-35731b8d5466 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:40.757330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b3e3-a2e8-4dc6-b64f-35731b8d5466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:40.768246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b3e3-a2e8-4dc6-b64f-35731b8d5466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:40.769821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b3e3-a2e8-4dc6-b64f-35731b8d5466 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:40.782639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 941/1873] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:41.031561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.033566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.084541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:41.144148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864beb25-04e9-4d4c-921f-587e0b0044f4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.149095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.150004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.150348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.150687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] PATH_INFO: `/v3/projects/2692e18eca0447739862c20f5527ed7d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.151877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2692e18eca0447739862c20f5527ed7d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.153993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.206572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2692e18eca0447739862c20f5527ed7d', 'request_id': 'req-07766d60-2c6a-4e17-aa5f-e8b676241c6a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.212772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2692e18eca0447739862c20f5527ed7d', 'request_id': 'req-07766d60-2c6a-4e17-aa5f-e8b676241c6a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.219588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2692e18eca0447739862c20f5527ed7d', 'request_id': 'req-07766d60-2c6a-4e17-aa5f-e8b676241c6a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.239972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] The token cache is being invalidate because project 2692e18eca0447739862c20f5527ed7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:41.242199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.242599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-07766d60-2c6a-4e17-aa5f-e8b676241c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.245105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 933/1874] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:41 2022] DELETE /identity/v3/projects/2692e18eca0447739862c20f5527ed7d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:41.253946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.253946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.253946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.271387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:41.272586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.334386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 942/1875] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:41.341702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.342027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.342310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.362633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:41.364082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.372074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 934/1876] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:41.377065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.387524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad650940-8c13-4d68-b71b-12a4ee1763a2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.392476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.393217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.393508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.393784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.394560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:41.394867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.396462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.411769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 943/1877] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:00:41.411901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.411901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.412712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.422388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239334d-2799-49e9-bc74-4fbfadedd415 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.423553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45aeff8-76fe-4cac-8d55-bb172d98abd9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.427760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.428078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.428340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.428654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.428774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.428924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.429019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.429280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] PATH_INFO: `/v3/users/088135830d88482390d35745733dfa15` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.429539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.430437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088135830d88482390d35745733dfa15)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.431480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.432344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.460284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 944/1878] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:41 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:41.464516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.480034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088135830d88482390d35745733dfa15'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.488330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088135830d88482390d35745733dfa15'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.500512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088135830d88482390d35745733dfa15'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.501720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088135830d88482390d35745733dfa15'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.508763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86c7b6a1-bc67-4dc2-a1d2-ac87382e9dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088135830d88482390d35745733dfa15'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:41.514980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 935/1879] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:41 2022] DELETE /identity/v3/users/088135830d88482390d35745733dfa15 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:41.521130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.521130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.521130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.529935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b8c81-b403-4a80-887e-4f68ffdedc82 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.540511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:41.541721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.546687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:41.567686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:41.591738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.592268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.592546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.592814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.593430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.595966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.614845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 936/1880] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:41.620358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 945/1881] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:41 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:41.621501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.622850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.624437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.626090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.634680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ebe30-dd8a-4943-af29-70af6b14e639 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.639307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.639906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.640194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.640478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.641131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.643037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.653345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 937/1882] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:41.657803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.667369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a6018-d281-4c4b-824d-16e69696ed47 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.672046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.673218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.673501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.673775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.674448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.676079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.683319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bed1a5-083a-427c-beb7-6f74de5f0115 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.685989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 938/1883] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:41.687319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb5f71-fe3a-4fb7-822f-7c2471a754e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.688017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb5f71-fe3a-4fb7-822f-7c2471a754e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.688341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb5f71-fe3a-4fb7-822f-7c2471a754e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.688674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb5f71-fe3a-4fb7-822f-7c2471a754e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.690532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.698574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb5f71-fe3a-4fb7-822f-7c2471a754e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.700039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb5f71-fe3a-4fb7-822f-7c2471a754e5 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.701179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57536f-c899-457e-ab0e-046b978c981e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:41.706430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:41.707075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.707358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.707631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] PATH_INFO: `/v3/projects/19ca2411742e4b14b8c563130ed2c939/users/59314bd6b9df474cb8cd92ede66eadde/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.709565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 946/1884] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:41.726564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ca2411742e4b14b8c563130ed2c939, user_id=59314bd6b9df474cb8cd92ede66eadde, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:41.729250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:41.747318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 939/1885] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:41 2022] PUT /identity/v3/projects/19ca2411742e4b14b8c563130ed2c939/users/59314bd6b9df474cb8cd92ede66eadde/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:41.758327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:41.758703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:41.759120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:41.803633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] MFA Rules not processed for user `59314bd6b9df474cb8cd92ede66eadde`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:41.805042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:41.885296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 947/1886] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:41.894387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:41.897281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:42.003891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30d75d-2cb7-4ea7-ba4b-63a25e12beff admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:42.008451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:42.009379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:42.010278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:42.010278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:42.030987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:42.033195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:42.043619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 940/1887] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:42.186089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:42.187823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:42.339565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b50d6-bce0-4d75-a745-1e5c32cb08da None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:42.347358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:42.348177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:42.348665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:42.349061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] PATH_INFO: `/v3/projects/0375f4b01b1a4a90acbb5a79c4537a05` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:42.350724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0375f4b01b1a4a90acbb5a79c4537a05)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:42.353562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:42.356109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:42.452280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0375f4b01b1a4a90acbb5a79c4537a05', 'request_id': 'req-fd1633d9-ba02-4945-a571-2c90cb528001'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.472230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0375f4b01b1a4a90acbb5a79c4537a05', 'request_id': 'req-fd1633d9-ba02-4945-a571-2c90cb528001'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.484145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0375f4b01b1a4a90acbb5a79c4537a05', 'request_id': 'req-fd1633d9-ba02-4945-a571-2c90cb528001'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.513720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] The token cache is being invalidate because project 0375f4b01b1a4a90acbb5a79c4537a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:42.514066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.514719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.517793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 948/1888] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:42 2022] DELETE /identity/v3/projects/0375f4b01b1a4a90acbb5a79c4537a05 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:42.521347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:42.537133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232754d-2832-4862-96df-4c5517899ef7 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:42.595642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:42.596271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:42.596585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:42.596849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] PATH_INFO: `/v3/users/6ffb8dbf5c7c411085cc42b2acd06596` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:42.598000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffb8dbf5c7c411085cc42b2acd06596)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:42.599878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:42.614625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:42.659016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffb8dbf5c7c411085cc42b2acd06596'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.666674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffb8dbf5c7c411085cc42b2acd06596'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.680011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffb8dbf5c7c411085cc42b2acd06596'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.680865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffb8dbf5c7c411085cc42b2acd06596'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.689516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f5636a9e-927c-4dca-8f32-f57d3b2eb3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffb8dbf5c7c411085cc42b2acd06596'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:42.696126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 941/1889] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:42 2022] DELETE /identity/v3/users/6ffb8dbf5c7c411085cc42b2acd06596 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:42.976419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:42.978612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:42.996519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1633d9-ba02-4945-a571-2c90cb528001 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.109707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.112183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.112469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.113947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] PATH_INFO: `/v3/projects/99e4b264441445edb4a35b21d2b2c9f2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.117824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e4b264441445edb4a35b21d2b2c9f2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.125729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.208879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e4b264441445edb4a35b21d2b2c9f2', 'request_id': 'req-40bcccd7-080e-4f6b-a590-64a6f9963da0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:43.217607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e4b264441445edb4a35b21d2b2c9f2', 'request_id': 'req-40bcccd7-080e-4f6b-a590-64a6f9963da0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:43.228388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e4b264441445edb4a35b21d2b2c9f2', 'request_id': 'req-40bcccd7-080e-4f6b-a590-64a6f9963da0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:43.253087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] The token cache is being invalidate because project 99e4b264441445edb4a35b21d2b2c9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:43.262225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:43.262225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-40bcccd7-080e-4f6b-a590-64a6f9963da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:43.262225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 949/1890] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:42 2022] DELETE /identity/v3/projects/99e4b264441445edb4a35b21d2b2c9f2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:43.262456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.262456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.262456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.288912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:43.290496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:43.366186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 942/1891] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:43.374148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.374148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.374694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.403064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:43.404281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:43.422690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 950/1892] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:43.426571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:43.452815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635c3b3-394c-4258-8cb7-b96025d55e86 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.476929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.482183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.482481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.482759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.483445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:43.483765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.485449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.515346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 943/1893] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:43.520458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:43.530774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf42e-501e-4098-b61f-c7245f17c18a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.537739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.539073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.539570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.540022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.540957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.543466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.560378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:43.589797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 951/1894] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:43 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:43.595343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:43.606456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd83706-eeea-4536-96c5-555bcff5c274 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.686178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.686838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.687408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.687712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.688872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.691722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.722984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 944/1895] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:43 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:43.724953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:43.739737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235de11-5602-46da-bc31-a7e19baabfe6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.745954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.746574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.746853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.747128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.747898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.750090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.763848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 952/1896] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:43.768595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:43.775114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:43.779261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c6f8e-f4b1-4875-beae-8fef2497bcf4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.785516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c1c7c-3cc9-4282-a58e-cf4c0706827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.786508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c1c7c-3cc9-4282-a58e-cf4c0706827b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.787003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c1c7c-3cc9-4282-a58e-cf4c0706827b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.787534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c1c7c-3cc9-4282-a58e-cf4c0706827b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.788178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c1c7c-3cc9-4282-a58e-cf4c0706827b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.789878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c1c7c-3cc9-4282-a58e-cf4c0706827b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.801036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 945/1897] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:43.805221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:43.818536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f083776-5c6c-47f0-a66a-fc3b74795849 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:43.831226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:43.832043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.833078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.833270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] PATH_INFO: `/v3/projects/6d24a4838dc043bdbadbb7b3d53287f1/users/f547b64675424852b9672efd5f88eb1a/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.849648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d24a4838dc043bdbadbb7b3d53287f1, user_id=f547b64675424852b9672efd5f88eb1a, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:43.852530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:43.868134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 953/1898] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:43 2022] PUT /identity/v3/projects/6d24a4838dc043bdbadbb7b3d53287f1/users/f547b64675424852b9672efd5f88eb1a/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:43.880086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:43.880496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:43.880670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:43.918005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] MFA Rules not processed for user `f547b64675424852b9672efd5f88eb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:43.919250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:44.068790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 946/1899] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:44.070182 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.151640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea26335-ebf0-4fc9-bba3-f2c57127dec7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.156391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.157036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.157322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.157589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.158256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.160162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.202303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 954/1900] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:44 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:44.207318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.222350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38adc5-d901-467d-8d3d-589d5bd65190 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.303020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.304140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.304704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.305036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.305692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.314396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.361408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 947/1901] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:44 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:44.362900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.387264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f181b-2079-4f9a-97f4-b22eee37cde4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.410971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.410971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.410971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.410971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.411393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.412028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.433876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 955/1902] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:44.437947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.450077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0452e3-1cee-4a3d-8cb2-0866d65a2def admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.455955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.456636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.456922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.457251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] PATH_INFO: `/v3/projects/38fe756e56944998b8e6c0bd087b14b8/users/fda6bb1169c74ac3a3598d41f4cfefd8/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.509422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fe756e56944998b8e6c0bd087b14b8, user_id=fda6bb1169c74ac3a3598d41f4cfefd8, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.519325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.557946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 948/1903] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:44 2022] PUT /identity/v3/projects/38fe756e56944998b8e6c0bd087b14b8/users/fda6bb1169c74ac3a3598d41f4cfefd8/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:44.559645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.573042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2a77-c290-4021-aeaa-0b04332e0c81 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.672481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.673944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.674423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.675579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.677337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.695647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.738397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 956/1904] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:44.742421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.756963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370848c-01b2-42ca-b657-402a08826797 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.763290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2523eb-fc27-4e9d-b737-ac128a46780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.764478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2523eb-fc27-4e9d-b737-ac128a46780d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.764927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2523eb-fc27-4e9d-b737-ac128a46780d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.765412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2523eb-fc27-4e9d-b737-ac128a46780d admin admin] PATH_INFO: `/v3/projects/38fe756e56944998b8e6c0bd087b14b8/users/fda6bb1169c74ac3a3598d41f4cfefd8/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.769015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2523eb-fc27-4e9d-b737-ac128a46780d admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fe756e56944998b8e6c0bd087b14b8, user_id=fda6bb1169c74ac3a3598d41f4cfefd8, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.774031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2523eb-fc27-4e9d-b737-ac128a46780d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.806440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 949/1905] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:44 2022] PUT /identity/v3/projects/38fe756e56944998b8e6c0bd087b14b8/users/fda6bb1169c74ac3a3598d41f4cfefd8/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:44.808313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:44.811408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:44.814055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.814348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.814622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.841884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6721-e5ec-4024-aef3-7735df804583 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:44.881597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] MFA Rules not processed for user `fda6bb1169c74ac3a3598d41f4cfefd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:44.882744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:44.883177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:44.915063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:44.920642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:44.921598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:44.921961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:44.923325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] PATH_INFO: `/v3/users/59314bd6b9df474cb8cd92ede66eadde` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:44.924447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59314bd6b9df474cb8cd92ede66eadde)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:44.926764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:44.984551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 950/1906] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:45.002177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:45.003883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:45.016931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59314bd6b9df474cb8cd92ede66eadde'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:45.035396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59314bd6b9df474cb8cd92ede66eadde'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:45.066202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59314bd6b9df474cb8cd92ede66eadde'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:45.067225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59314bd6b9df474cb8cd92ede66eadde'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:45.079467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-313154b6-649d-426b-9627-170144b1afc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59314bd6b9df474cb8cd92ede66eadde'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:45.089969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 957/1907] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:44 2022] DELETE /identity/v3/users/59314bd6b9df474cb8cd92ede66eadde => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:45.094915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:45.095281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:45.095703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:45.104010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca072f5-fc27-45ea-9f7e-cfaab181cb99 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:45.121269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:45.122541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:45.196467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 958/1908] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:45.202885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:45.203709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:45.204158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:45.204433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:45.207348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:45.207745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:45.211038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:45.286843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:45.288521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:45.292537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:45.298595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 951/1909] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:45.321017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7d963-f15d-4f3e-967c-f8635a68b8cc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:45.325519 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:45.326417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:45.326935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:45.327309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:45.337630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:45.339255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:45.352014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 959/1910] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:45.763989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:45.765607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:45.779221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:45.780849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:45.800176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3893f27-9174-4fd5-9258-9a4884588a5d service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:45.804967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0fef-8844-4c82-a5e3-512792f01b1b service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:45.826657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:45.829460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:45.830403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:45.830829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] PATH_INFO: `/v3/users/8ae2be3e5db640859f5a4fc8f9229756` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:45.832228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae2be3e5db640859f5a4fc8f9229756)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:45.834361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:45.842396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:45.847559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:45.847743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:45.848028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] PATH_INFO: `/v3/projects/19ca2411742e4b14b8c563130ed2c939` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:45.852747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ca2411742e4b14b8c563130ed2c939)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:45.856410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:45.937223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:45.981929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae2be3e5db640859f5a4fc8f9229756'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:45.995047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae2be3e5db640859f5a4fc8f9229756'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.005380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ca2411742e4b14b8c563130ed2c939', 'request_id': 'req-a1c257b8-79db-4c8a-be78-472ce22109dd'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.024916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae2be3e5db640859f5a4fc8f9229756'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.025160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ca2411742e4b14b8c563130ed2c939', 'request_id': 'req-a1c257b8-79db-4c8a-be78-472ce22109dd'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.029277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae2be3e5db640859f5a4fc8f9229756'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.032956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ca2411742e4b14b8c563130ed2c939', 'request_id': 'req-a1c257b8-79db-4c8a-be78-472ce22109dd'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.053252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] The token cache is being invalidate because project 19ca2411742e4b14b8c563130ed2c939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:46.053547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.056654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-adbd5ec9-c58c-4819-bf7e-84d7325913d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae2be3e5db640859f5a4fc8f9229756'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.058073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1c257b8-79db-4c8a-be78-472ce22109dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:46.064775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 960/1911] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:45 2022] DELETE /identity/v3/projects/19ca2411742e4b14b8c563130ed2c939 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:46.066536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb9e28-ecfd-450e-bfcf-cd00e9dfe5c1 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.066856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb9e28-ecfd-450e-bfcf-cd00e9dfe5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.067133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb9e28-ecfd-450e-bfcf-cd00e9dfe5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.079694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 952/1912] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:45 2022] DELETE /identity/v3/users/8ae2be3e5db640859f5a4fc8f9229756 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:46.089274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.089567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.089934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.100275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a5eb9e28-ecfd-450e-bfcf-cd00e9dfe5c1 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:46.103645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a5eb9e28-ecfd-450e-bfcf-cd00e9dfe5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:46.137773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:46.139195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:46.243757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 961/1913] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:46.245846 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.246156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.248046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.278419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 953/1914] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:46.287968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:46.289299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:46.306543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 962/1915] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:46.308127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.320880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dce8fe-cf1d-4070-9c4a-dfafb744d1fc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.330502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.331978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:46.341272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.342373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.343461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.343971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.345062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:46.345793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.348357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.380311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 954/1916] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:46.396016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.408957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb607c-0920-4d6f-9b80-9f415052eb43 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.416487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.417395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.417849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.418551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.419365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.421527 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.433813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f2844-bc97-4e8d-b9f1-88e834d1455a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.444691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.447446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.447731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.448071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.459911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.460804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 955/1917] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:46 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:46.464739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.465689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.476719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 963/1918] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:46.551855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3437-faed-4250-b540-eb8e335cf61b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.556744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.557425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.557784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.558147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.558847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.560839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.576764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:46.584170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 956/1919] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:46 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:46.588686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.598572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447264e-bd8b-43f4-aec9-8b2333da5de4 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.603755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.604351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.604630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.604904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.605537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.607158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.609343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:46.622868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 964/1920] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:46.624255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.635185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066f605-d25b-4d21-b340-f455db8a5d4b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.639800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19864286-abff-4800-b5cb-5e8e69bcd433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.640430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19864286-abff-4800-b5cb-5e8e69bcd433 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.640713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19864286-abff-4800-b5cb-5e8e69bcd433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.640985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19864286-abff-4800-b5cb-5e8e69bcd433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.642461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19864286-abff-4800-b5cb-5e8e69bcd433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.643385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19864286-abff-4800-b5cb-5e8e69bcd433 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.653879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 957/1921] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:46.659501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.670702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e21f4-d52e-4df2-bd7e-eb591b7cf8db admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.676562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.677300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.677687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.678021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] PATH_INFO: `/v3/projects/eddca7f4673e49dc93a6e1c15e78c521/users/d80d5192d7224516bdff0af8c99f5cea/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.702399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddca7f4673e49dc93a6e1c15e78c521, user_id=d80d5192d7224516bdff0af8c99f5cea, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.705298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.718292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 965/1922] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:46 2022] PUT /identity/v3/projects/eddca7f4673e49dc93a6e1c15e78c521/users/d80d5192d7224516bdff0af8c99f5cea/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:46.728529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.728771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.729064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.759516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] MFA Rules not processed for user `d80d5192d7224516bdff0af8c99f5cea`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:46.760771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:46.828198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 958/1923] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:46.839286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:46.841935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:46.924359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d15f9-0d05-48f4-9c48-bcd7eef3d5af admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:46.934719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:46.935576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:46.935864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:46.936149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:46.948944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:46.950446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:46.968186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 966/1924] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:47.030737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:47.033349 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:47.164193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa4478-139c-4f31-b1d9-24fd8e05d6c5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:47.169182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:47.169750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:47.170072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:47.170348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] PATH_INFO: `/v3/projects/cec8e339d755416fbdc67477e6022dc5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:47.172066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec8e339d755416fbdc67477e6022dc5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:47.174220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:47.244243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec8e339d755416fbdc67477e6022dc5', 'request_id': 'req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.250336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec8e339d755416fbdc67477e6022dc5', 'request_id': 'req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.257749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec8e339d755416fbdc67477e6022dc5', 'request_id': 'req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.277328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] The token cache is being invalidate because project cec8e339d755416fbdc67477e6022dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:47.277868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.278847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.281092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 959/1925] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:47 2022] DELETE /identity/v3/projects/cec8e339d755416fbdc67477e6022dc5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:47.285693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:47.299783 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffe468-2ecc-4946-89fa-3bbaf0a46d3c service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:47.406748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:47.409034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:47.409318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:47.411350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] PATH_INFO: `/v3/users/e6224cdb4a504f5ba3c1b5773c4d3ab9` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:47.412824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6224cdb4a504f5ba3c1b5773c4d3ab9)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:47.416942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:47.520008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6224cdb4a504f5ba3c1b5773c4d3ab9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.533679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6224cdb4a504f5ba3c1b5773c4d3ab9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.580607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6224cdb4a504f5ba3c1b5773c4d3ab9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.582755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6224cdb4a504f5ba3c1b5773c4d3ab9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.593212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:47.595823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eae2d4cd-731b-4051-ab5b-164ba211d717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6224cdb4a504f5ba3c1b5773c4d3ab9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:47.612556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 967/1926] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:47 2022] DELETE /identity/v3/users/e6224cdb4a504f5ba3c1b5773c4d3ab9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:48.019107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.023078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:48.049065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d11d54-fafa-4e03-b536-ca6c0f6f8a8f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.061535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:48.143902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.144547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.144834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.145105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] PATH_INFO: `/v3/projects/e89112095a8b4d499bbeb585c7f52ac8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.146312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89112095a8b4d499bbeb585c7f52ac8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.148219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.198784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89112095a8b4d499bbeb585c7f52ac8', 'request_id': 'req-405f492a-d453-41e8-b7bc-32cf098d857d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:48.204301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89112095a8b4d499bbeb585c7f52ac8', 'request_id': 'req-405f492a-d453-41e8-b7bc-32cf098d857d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:48.211113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89112095a8b4d499bbeb585c7f52ac8', 'request_id': 'req-405f492a-d453-41e8-b7bc-32cf098d857d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:48.224492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] The token cache is being invalidate because project e89112095a8b4d499bbeb585c7f52ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:48.224785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:48.225610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-405f492a-d453-41e8-b7bc-32cf098d857d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:48.227135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 960/1927] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:48 2022] DELETE /identity/v3/projects/e89112095a8b4d499bbeb585c7f52ac8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:48.231970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.232445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.233082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.255538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:48.256852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:48.321350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 968/1928] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:48.329558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.329846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.330170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.353289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:48.355483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:48.368620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 961/1929] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:48.372953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.383214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f2f1-d03d-4fc9-85d0-07abb952ad7b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.388036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.388735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.389076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.389414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.390174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:48.390548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.392444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.404912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 969/1930] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:48.409976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.419927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e4f68-cba7-4639-b7ab-5e24a076dad6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.425324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.426076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.426293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.426642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.427390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.429326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.462821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 962/1931] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:48.467427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.478497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a0402-4b41-45be-ac18-af1a1d1f7c57 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.560612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d699f-a2f1-43c6-ae07-e89ad8c676e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.561269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d699f-a2f1-43c6-ae07-e89ad8c676e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.561551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d699f-a2f1-43c6-ae07-e89ad8c676e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.561826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d699f-a2f1-43c6-ae07-e89ad8c676e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.562526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d699f-a2f1-43c6-ae07-e89ad8c676e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.564486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d699f-a2f1-43c6-ae07-e89ad8c676e8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.593338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 970/1932] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/users => generated 279 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:48.598059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.622058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7513e7-e573-442a-88e6-284de6bb77b8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.628861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb43034-e541-4f1f-a4f9-baf26d2c58cb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.629218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb43034-e541-4f1f-a4f9-baf26d2c58cb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.629565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb43034-e541-4f1f-a4f9-baf26d2c58cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.630611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.631533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.632184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.632968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.634196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.636098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.652581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 963/1933] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:48.654439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-beb43034-e541-4f1f-a4f9-baf26d2c58cb None None] MFA Rules not processed for user `d80d5192d7224516bdff0af8c99f5cea`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:48.656278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-beb43034-e541-4f1f-a4f9-baf26d2c58cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:48.656511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.684623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5e4c3-7e7b-4e3d-8bea-d63c9dc171a7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.692051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.694376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.696448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.696961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.697954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.700965 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.728709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 964/1934] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:48.731887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:48.734134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 971/1935] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:48.748093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad215e33-77fe-45e0-ab39-a30d5621a391 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:48.755536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:48.756702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.756994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.757270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] PATH_INFO: `/v3/projects/1a59516a57b54cb8a6b2b50b500f09e7/users/bba4ac3d238343c78bf23bbd76258a39/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.790170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a59516a57b54cb8a6b2b50b500f09e7, user_id=bba4ac3d238343c78bf23bbd76258a39, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:48.793050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:48.817533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 965/1936] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:48 2022] PUT /identity/v3/projects/1a59516a57b54cb8a6b2b50b500f09e7/users/bba4ac3d238343c78bf23bbd76258a39/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:48.827745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:48.828030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:48.830380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:48.887088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] MFA Rules not processed for user `bba4ac3d238343c78bf23bbd76258a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:48.888590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:48.898915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:48.985854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 972/1937] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:48.990673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:49.034249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:49.119951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba670-d958-4645-a0bd-4fc5dbd69b0a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:49.125158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:49.126578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.126870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.127152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.127802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:49.132239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:49.192721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 966/1938] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:48 2022] POST /identity/v3/projects => generated 355 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:49.196434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:49.237057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab7a02-600b-4572-a2ef-e5d0509f6014 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:49.357580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:49.359365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.359980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.360386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.361226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:49.363579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:49.403445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 973/1939] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:49 2022] POST /identity/v3/users => generated 284 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:49.405137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:49.423317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4bfe6-e09c-4b9c-aede-3bc4d1ff94b6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:49.428646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:49.429437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.430201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.431228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.432054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:49.433817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:49.451587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 967/1940] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:49.456318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:49.468187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb66706-2771-4714-99fa-bd6380981ce0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:49.482570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:49.482570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.482570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.482570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] PATH_INFO: `/v3/projects/4a986e884d184b3581ac1fd325d1573f/users/b9f819c39f22473fbdd1b08c71cab3d4/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.514026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a986e884d184b3581ac1fd325d1573f, user_id=b9f819c39f22473fbdd1b08c71cab3d4, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:49.518258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:49.546453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 974/1941] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:49 2022] PUT /identity/v3/projects/4a986e884d184b3581ac1fd325d1573f/users/b9f819c39f22473fbdd1b08c71cab3d4/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:49.551080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:49.569172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd468d-ad0e-49fe-8b48-346d747f2ac1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:49.716403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efafaf1-ead3-4acc-85b7-8cd261266be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:49.717261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efafaf1-ead3-4acc-85b7-8cd261266be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.717796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efafaf1-ead3-4acc-85b7-8cd261266be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.718073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efafaf1-ead3-4acc-85b7-8cd261266be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.718921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efafaf1-ead3-4acc-85b7-8cd261266be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:49.720810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efafaf1-ead3-4acc-85b7-8cd261266be4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:49.753935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 968/1942] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:49.757432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:49.771212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc3b9-9c40-4162-b03b-3455ebf1c2d5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:49.780307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:49.781060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.785774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.786308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] PATH_INFO: `/v3/projects/4a986e884d184b3581ac1fd325d1573f/users/b9f819c39f22473fbdd1b08c71cab3d4/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.792022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a986e884d184b3581ac1fd325d1573f, user_id=b9f819c39f22473fbdd1b08c71cab3d4, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:49.801581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:49.842778 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 975/1943] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:49 2022] PUT /identity/v3/projects/4a986e884d184b3581ac1fd325d1573f/users/b9f819c39f22473fbdd1b08c71cab3d4/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:49.853067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:49.853523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:49.853962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:49.923851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] MFA Rules not processed for user `b9f819c39f22473fbdd1b08c71cab3d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:49.925328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:50.031722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 969/1944] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:50.044565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:50.046696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:50.117442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7516c-e2b8-4095-b289-0f919dc58a1f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:50.121053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:50.121672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:50.121972 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:50.122245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:50.133366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:50.134925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:50.146823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 976/1945] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:50.241372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:50.244451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:50.260677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c9314-16f8-4cad-9c0c-a49f8f3f9eda None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:50.265081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:50.265706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:50.266211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:50.266472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:50.282253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:50.337252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:50.338690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:50.360653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 970/1946] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:50.369708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:50.376305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:50.382643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccacbb3-40b6-401c-b6e7-911f99c0747c service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:50.388715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:50.390828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:50.392343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:50.393066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:50.395818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:50.490036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:50.491691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:50.507354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 977/1947] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:54.571875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:54.573453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:54.665924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4f723-868b-4ef8-962a-201947c5f6f5 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:54.665924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:54.665924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:54.665924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:54.666520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] PATH_INFO: `/v3/users/2b7e96d63d7446cfa3bc80784944c474` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:54.666520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7e96d63d7446cfa3bc80784944c474)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:54.666520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:54.717150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7e96d63d7446cfa3bc80784944c474'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:54.726576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7e96d63d7446cfa3bc80784944c474'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:54.740492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7e96d63d7446cfa3bc80784944c474'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:54.741134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7e96d63d7446cfa3bc80784944c474'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:54.748365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7e96d63d7446cfa3bc80784944c474'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:54.756873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 971/1948] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:54 2022] DELETE /identity/v3/users/2b7e96d63d7446cfa3bc80784944c474 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:54.966691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:54.984077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d2b1-f5de-4569-bb0c-fd72d944a460 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.066705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:55.067344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.067726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.067876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] PATH_INFO: `/v3/projects/181ad9c9850742ea814a39b97e61c79b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.069472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=181ad9c9850742ea814a39b97e61c79b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:55.071588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:55.142716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181ad9c9850742ea814a39b97e61c79b', 'request_id': 'req-92e9dc0b-1904-448d-b971-3abbe78f4eda'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.148507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181ad9c9850742ea814a39b97e61c79b', 'request_id': 'req-92e9dc0b-1904-448d-b971-3abbe78f4eda'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.154789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181ad9c9850742ea814a39b97e61c79b', 'request_id': 'req-92e9dc0b-1904-448d-b971-3abbe78f4eda'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.168976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] The token cache is being invalidate because project 181ad9c9850742ea814a39b97e61c79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:55.169254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.169881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.171378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 978/1949] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:54 2022] DELETE /identity/v3/projects/181ad9c9850742ea814a39b97e61c79b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:55.176055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:55.177454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:55.190671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae723d-de2f-40ee-a58b-94f2cb7eafbd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.223093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:55.250550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:55.251220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.251545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.251850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] PATH_INFO: `/v3/users/48f3c94371764ea4bf2418b351801e3f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.253120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f3c94371764ea4bf2418b351801e3f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:55.255151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:55.300361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f3c94371764ea4bf2418b351801e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.308074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f3c94371764ea4bf2418b351801e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.320173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f3c94371764ea4bf2418b351801e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.320804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f3c94371764ea4bf2418b351801e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.327616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e34b93c-0cdb-4b99-9506-4fd592bf0bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f3c94371764ea4bf2418b351801e3f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.333518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 972/1950] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:55 2022] DELETE /identity/v3/users/48f3c94371764ea4bf2418b351801e3f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:55.507260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:55.508915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:55.529968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9dc0b-1904-448d-b971-3abbe78f4eda admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.555847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:55.590197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:55.590838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.591124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.591400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] PATH_INFO: `/v3/projects/e8285ac516b44017820f6d41d4d08b2c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.592523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8285ac516b44017820f6d41d4d08b2c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:55.595048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:55.640409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8285ac516b44017820f6d41d4d08b2c', 'request_id': 'req-277c0d08-54df-4d0c-aaff-b65d5b923e9b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.646338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8285ac516b44017820f6d41d4d08b2c', 'request_id': 'req-277c0d08-54df-4d0c-aaff-b65d5b923e9b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.652500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8285ac516b44017820f6d41d4d08b2c', 'request_id': 'req-277c0d08-54df-4d0c-aaff-b65d5b923e9b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.665231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] The token cache is being invalidate because project e8285ac516b44017820f6d41d4d08b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:55.665527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.666181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-277c0d08-54df-4d0c-aaff-b65d5b923e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:55.667786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 979/1951] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:55 2022] DELETE /identity/v3/projects/e8285ac516b44017820f6d41d4d08b2c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:55.672973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.673480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.673582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.693481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:55.694919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:55.759152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 973/1952] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:55.768803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.769097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.769372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.790769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:55.792028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:55.800415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 980/1953] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:55.804876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:55.817141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714fd4-d840-4b21-baca-e9ebba1a8f4e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.822081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:55.822671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.822950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.823223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.823902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:55.824204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:55.825843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:55.840044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 974/1954] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:00:55.844350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:55.853818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a02862-ef92-414a-8465-a35b96f06d64 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.858775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:55.860344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.860629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.860909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.861545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:55.863785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:55.892690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 981/1955] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:55 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:55.896687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:55.906804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c86dbe-4c82-4ee9-8bd5-4b683afec06f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.917433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:55.919250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:55.951100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:55.969842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:55.970548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:55.970839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:55.971117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:55.971779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:55.973696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:55.992774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac48d1-fe86-4819-8385-3ffc4a48414d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:55.999777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.000787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 975/1956] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:55 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:00:56.002573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.002870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.004653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.006066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] PATH_INFO: `/v3/users/f547b64675424852b9672efd5f88eb1a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.007632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f547b64675424852b9672efd5f88eb1a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.010156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.018476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b5bf8-1d09-4300-ba35-b1320c9c1885 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.023578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.024307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.024465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.024741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.025458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.027336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.039006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 976/1957] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:56.043086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.052236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:56.054133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438ef15-e3b7-4a32-97df-bb99debeca63 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.059024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.059616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.059944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.060220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.060868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.062481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.067953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f547b64675424852b9672efd5f88eb1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.073214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 977/1958] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:00:56.075436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f547b64675424852b9672efd5f88eb1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.077631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.089724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e114d-56e4-4f57-85f5-bb2f054de6de admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.091313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f547b64675424852b9672efd5f88eb1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.092075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f547b64675424852b9672efd5f88eb1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.095195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5ce06-494d-4069-a767-74eb848d9c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.095886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5ce06-494d-4069-a767-74eb848d9c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.096204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5ce06-494d-4069-a767-74eb848d9c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.096485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5ce06-494d-4069-a767-74eb848d9c89 admin admin] PATH_INFO: `/v3/projects/934382300f5f4af98706b721ce84ae36/users/67f6f1b1653246fb82e9d48813db8701/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.102435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a3432504-fdd1-4f80-a7e2-b2aed6e0e230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f547b64675424852b9672efd5f88eb1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.110080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 982/1959] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:55 2022] DELETE /identity/v3/users/f547b64675424852b9672efd5f88eb1a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:56.114153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5ce06-494d-4069-a767-74eb848d9c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934382300f5f4af98706b721ce84ae36, user_id=67f6f1b1653246fb82e9d48813db8701, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.114713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.115007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.115284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.116839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5ce06-494d-4069-a767-74eb848d9c89 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.144411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 978/1960] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:56 2022] PUT /identity/v3/projects/934382300f5f4af98706b721ce84ae36/users/67f6f1b1653246fb82e9d48813db8701/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:00:56.144546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:56.144546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:56.145569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.145861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.146157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.165999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:56.167194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:56.215847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 983/1961] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:56.228022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.232465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:56.245334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 979/1962] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:56.256017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.257475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:56.296737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac7bcd-b62d-4db0-aa8d-bbaf99df2b83 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.301897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.302565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.302843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.303115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.312773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.315397 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.327462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 984/1963] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:56.332339 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd4cc7-2a43-42e1-b52b-90c7478107a1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.336267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.336891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.337228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.337441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.351640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.354166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.369949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 980/1964] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:56.807685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.809258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:56.824168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dbc74-8d87-43a1-be62-eb7e2b25c7e1 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.829870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:56.830543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:56.830854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:56.831238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] PATH_INFO: `/v3/projects/6d24a4838dc043bdbadbb7b3d53287f1` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:56.832375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d24a4838dc043bdbadbb7b3d53287f1)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:56.834369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:56.904263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:56.908995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d24a4838dc043bdbadbb7b3d53287f1', 'request_id': 'req-039a19bb-1850-4227-bbbc-3ceec41677c1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.919870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d24a4838dc043bdbadbb7b3d53287f1', 'request_id': 'req-039a19bb-1850-4227-bbbc-3ceec41677c1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.928699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d24a4838dc043bdbadbb7b3d53287f1', 'request_id': 'req-039a19bb-1850-4227-bbbc-3ceec41677c1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.947515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] The token cache is being invalidate because project 6d24a4838dc043bdbadbb7b3d53287f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:56.947880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.948712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-039a19bb-1850-4227-bbbc-3ceec41677c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:56.953854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 985/1965] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:56 2022] DELETE /identity/v3/projects/6d24a4838dc043bdbadbb7b3d53287f1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:56.955043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:56.965787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51ce89-01f5-4755-a3e1-82a585ebfadc service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:56.998394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:57.058704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:57.060528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:57.060901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:57.061214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] PATH_INFO: `/v3/users/fda6bb1169c74ac3a3598d41f4cfefd8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:57.063347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda6bb1169c74ac3a3598d41f4cfefd8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:57.065359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:57.119348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda6bb1169c74ac3a3598d41f4cfefd8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.127652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda6bb1169c74ac3a3598d41f4cfefd8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.141674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda6bb1169c74ac3a3598d41f4cfefd8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.142379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda6bb1169c74ac3a3598d41f4cfefd8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.152085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda6bb1169c74ac3a3598d41f4cfefd8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.158955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 981/1966] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:56 2022] DELETE /identity/v3/users/fda6bb1169c74ac3a3598d41f4cfefd8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:57.202717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:57.203189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:57.203270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:57.245303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] MFA Rules not processed for user `67f6f1b1653246fb82e9d48813db8701`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:57.247024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:57.330365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 986/1967] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:57.339854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:57.341379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:57.437037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56c645-0885-44a1-867a-b6f4669f974d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:57.441489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916983ca-bc45-4ad6-aa3c-84c112b535c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:57.442241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916983ca-bc45-4ad6-aa3c-84c112b535c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:57.442570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916983ca-bc45-4ad6-aa3c-84c112b535c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:57.442925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916983ca-bc45-4ad6-aa3c-84c112b535c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:57.455169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916983ca-bc45-4ad6-aa3c-84c112b535c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:57.457265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916983ca-bc45-4ad6-aa3c-84c112b535c2 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:57.469323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 982/1968] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:57.565122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:57.566423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:57.677888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4d099-03ca-4546-aba7-f4647cf1e75b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:57.684773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:57.685824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:57.686308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:57.686894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] PATH_INFO: `/v3/projects/38fe756e56944998b8e6c0bd087b14b8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:57.688955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38fe756e56944998b8e6c0bd087b14b8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:57.691793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:57.769885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38fe756e56944998b8e6c0bd087b14b8', 'request_id': 'req-453ab0b2-e358-4a9d-b364-b5e38e100bd5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.777998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38fe756e56944998b8e6c0bd087b14b8', 'request_id': 'req-453ab0b2-e358-4a9d-b364-b5e38e100bd5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.786920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38fe756e56944998b8e6c0bd087b14b8', 'request_id': 'req-453ab0b2-e358-4a9d-b364-b5e38e100bd5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.790725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:57.806884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] The token cache is being invalidate because project 38fe756e56944998b8e6c0bd087b14b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:57.807458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.808683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-453ab0b2-e358-4a9d-b364-b5e38e100bd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:57.811062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 987/1969] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:57 2022] DELETE /identity/v3/projects/38fe756e56944998b8e6c0bd087b14b8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:57.820578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:57.820927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:57.821183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:57.844756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:57.846009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:57.957178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:57.992586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 983/1970] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:00:58.002622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.002979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.003315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.029522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:58.032068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:58.042019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 988/1971] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:58.045153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.055748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893d142-dc1c-47d9-9fb7-4c79068e7a1e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.061088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.061697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.062010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.062290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.063042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:00:58.063359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.065238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.081855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 984/1972] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:00:58.086333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.096777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8bdb-c4b1-41f6-bcde-11884c46f4a8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.102104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.102733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.103034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.103310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.103963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.106369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.143023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 989/1973] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:00:58 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:58.147188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.168354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548f9a4-3a52-4fc4-8cc3-83006125934e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.255284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.256077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.256364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.256669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.257326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.259411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.287622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 985/1974] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:00:58 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:00:58.289542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.310857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dea78-9bf9-47a4-8b38-2d1f88b410a3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.340765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.341443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.341793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.342184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.343016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.345491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.360274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 990/1975] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:58.365123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.380920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91793d-7d75-4a8c-9cf8-b529c2ef03bf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.385961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034855f0-c10d-43bb-8da9-df3a1961fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.386567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034855f0-c10d-43bb-8da9-df3a1961fbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.386853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034855f0-c10d-43bb-8da9-df3a1961fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.387132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034855f0-c10d-43bb-8da9-df3a1961fbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.388066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034855f0-c10d-43bb-8da9-df3a1961fbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.389668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034855f0-c10d-43bb-8da9-df3a1961fbc2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.401308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 986/1976] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:00:58.405574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.415868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd82c-533b-45ad-bac2-71351ce5f5f8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.420404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.421045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.421327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.421603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] PATH_INFO: `/v3/projects/9b23073e7ce9447996e9526acc38293e/users/94fd3d6b30e04c84a7b36019f597245d/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.435685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b23073e7ce9447996e9526acc38293e, user_id=94fd3d6b30e04c84a7b36019f597245d, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.438439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.450989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 991/1977] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:00:58 2022] PUT /identity/v3/projects/9b23073e7ce9447996e9526acc38293e/users/94fd3d6b30e04c84a7b36019f597245d/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:58.463795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.464081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.464358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.493973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] MFA Rules not processed for user `94fd3d6b30e04c84a7b36019f597245d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:58.495198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:58.558825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 987/1978] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:58.570898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.572691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:58.632890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3dbd-bd68-4005-bc9b-9263c3b06532 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.636801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bddd8d-1b5e-4ded-a059-88d13f4da71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.637415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bddd8d-1b5e-4ded-a059-88d13f4da71d service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.637703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bddd8d-1b5e-4ded-a059-88d13f4da71d service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.638008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bddd8d-1b5e-4ded-a059-88d13f4da71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.648153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bddd8d-1b5e-4ded-a059-88d13f4da71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.650620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bddd8d-1b5e-4ded-a059-88d13f4da71d service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.660722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 992/1979] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:00:58.682435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:58.750650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:58.785163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315597-73ba-464d-87ad-13218e6dbc8d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:58.794624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:58.795385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.795758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.796102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] PATH_INFO: `/v3/users/94fd3d6b30e04c84a7b36019f597245d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.797688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fd3d6b30e04c84a7b36019f597245d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:58.800986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:58.880854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fd3d6b30e04c84a7b36019f597245d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:58.890552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fd3d6b30e04c84a7b36019f597245d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:58.906152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fd3d6b30e04c84a7b36019f597245d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:58.906991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fd3d6b30e04c84a7b36019f597245d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:58.914333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-294414b6-4bf1-4a25-b4a5-1adfc0a33a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fd3d6b30e04c84a7b36019f597245d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:58.923839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 988/1980] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:00:58 2022] DELETE /identity/v3/users/94fd3d6b30e04c84a7b36019f597245d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:58.928233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:58.928608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:58.928962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:58.948331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:58.949543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:59.020674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 993/1981] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:59.027602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:59.028068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:59.028162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:59.032564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:59.035195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:59.049669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] MFA Rules not processed for user `67f6f1b1653246fb82e9d48813db8701`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:59.051032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:59.083489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:59.127899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c14b2c-b6a5-4491-aa20-d86dc8b2fc39 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:59.132156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b497fd-8edf-45c7-836c-48d19084da93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:59.132771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b497fd-8edf-45c7-836c-48d19084da93 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:59.133053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b497fd-8edf-45c7-836c-48d19084da93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:59.133327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b497fd-8edf-45c7-836c-48d19084da93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:59.143927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b497fd-8edf-45c7-836c-48d19084da93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:59.145184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b497fd-8edf-45c7-836c-48d19084da93 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:59.157698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 989/1982] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:59.162028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 994/1983] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:00:59.688562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:00:59.690240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:59.714616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7d67-0c70-46ab-b1da-cf32008cecbd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:00:59.723387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:00:59.724115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:59.724507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:59.724923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] PATH_INFO: `/v3/projects/9b23073e7ce9447996e9526acc38293e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:59.726199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b23073e7ce9447996e9526acc38293e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:00:59.729399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:00:59.789930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b23073e7ce9447996e9526acc38293e', 'request_id': 'req-3f245883-a9d6-4b93-bea2-aba18d60a650'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:59.796098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b23073e7ce9447996e9526acc38293e', 'request_id': 'req-3f245883-a9d6-4b93-bea2-aba18d60a650'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:59.804229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b23073e7ce9447996e9526acc38293e', 'request_id': 'req-3f245883-a9d6-4b93-bea2-aba18d60a650'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:59.807364 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:00:59.825269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] The token cache is being invalidate because project 9b23073e7ce9447996e9526acc38293e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:00:59.826008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:59.827379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3f245883-a9d6-4b93-bea2-aba18d60a650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:00:59.829877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 990/1984] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:00:59 2022] DELETE /identity/v3/projects/9b23073e7ce9447996e9526acc38293e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:00:59.834961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:59.835417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:59.835645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:00:59.870381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:00:59.871675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:00:59.991449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 995/1985] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:00:59.992178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:00:59.994317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:00:59.994317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.036769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:00.038182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:00.058353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 991/1986] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:00.066647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.080920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb537832-8c0a-47b5-8228-269afa61dbff None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.110934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.111858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.112212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.112556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.113705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:00.114111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.138751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.166842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 996/1987] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:00.171850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.184244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb788-948a-476e-92b1-ec205acf1bc2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.193104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.193727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.194046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.194322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.194960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.196887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.228331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 992/1988] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:00 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:00.232324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.242890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724f0bd-7299-4650-ba8d-bf483b5f78a4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.305403 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.306077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.306388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.306663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.307293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.309289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.332337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 997/1989] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:00 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:00.336409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.347271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72357-cfab-4111-853c-376aec695a02 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.352073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.352691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.352970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.353240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.354057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.355623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.366788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 993/1990] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:00.371205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.382093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3b775-a3ef-477c-92f2-59d32433ae0c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.387006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4abcd-5fa6-4eda-9309-72d97e8213fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.387601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4abcd-5fa6-4eda-9309-72d97e8213fa admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.387883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4abcd-5fa6-4eda-9309-72d97e8213fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.388156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4abcd-5fa6-4eda-9309-72d97e8213fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.388795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4abcd-5fa6-4eda-9309-72d97e8213fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.390434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4abcd-5fa6-4eda-9309-72d97e8213fa admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.402386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 998/1991] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:00.406492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.416322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fa41-6679-4e88-8141-fc055cbda7e4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.420845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.421491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.421775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.422118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] PATH_INFO: `/v3/projects/641f977a99cc483fbe08d5b6fdd7b46c/users/339a40a7ae1b4eb684c2952d5e7b6ff8/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.437029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=641f977a99cc483fbe08d5b6fdd7b46c, user_id=339a40a7ae1b4eb684c2952d5e7b6ff8, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.440211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.453981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 994/1992] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:00 2022] PUT /identity/v3/projects/641f977a99cc483fbe08d5b6fdd7b46c/users/339a40a7ae1b4eb684c2952d5e7b6ff8/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:00.464106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.464487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.464796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.494434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] MFA Rules not processed for user `339a40a7ae1b4eb684c2952d5e7b6ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:00.495663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:00.525229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:00.563568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 999/1993] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:00.568138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.628941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762df3a-80b2-4f28-9dcf-acea291a1e59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.633702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.634335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.634625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.634908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.635550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.637444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.668908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 995/1994] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:00 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:00.670074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.682267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681d27-68de-40f7-8f3a-6d02b54a9c07 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.753018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.753623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.753918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.754203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.754844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.757029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.781502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1000/1995] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:00 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:00.785762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.797789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abb988-1f5a-45f1-a646-fcec19b08990 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.803388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.804049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.804311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.804585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.805232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.806921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.824765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 996/1996] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:00.825878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.835855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d94e-69f2-417d-8452-1aa2b99afe3e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.840648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.841312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.841635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.842093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] PATH_INFO: `/v3/projects/ed1e5a35354e4701990287503dcf2c0a/users/45e5eac68e294a6b9f6cf624b097753a/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.860127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1e5a35354e4701990287503dcf2c0a, user_id=45e5eac68e294a6b9f6cf624b097753a, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.862699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.883821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1001/1997] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:00 2022] PUT /identity/v3/projects/ed1e5a35354e4701990287503dcf2c0a/users/45e5eac68e294a6b9f6cf624b097753a/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:00.888286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:00.898778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f00a8-7289-46af-9862-70a4aaa15f6b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:00.970138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4c0e3-bb31-4f3f-8c04-7b3bb2660fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:00.970994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4c0e3-bb31-4f3f-8c04-7b3bb2660fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:00.971305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4c0e3-bb31-4f3f-8c04-7b3bb2660fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:00.971725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4c0e3-bb31-4f3f-8c04-7b3bb2660fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:00.972528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4c0e3-bb31-4f3f-8c04-7b3bb2660fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:00.974360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4c0e3-bb31-4f3f-8c04-7b3bb2660fd9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:00.982469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:00.991249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 997/1998] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:01.005009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:01.016982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f08ac5-646b-498d-8047-b39789c45613 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:01.021682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:01.022477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:01.022756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:01.023027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] PATH_INFO: `/v3/projects/ed1e5a35354e4701990287503dcf2c0a/users/45e5eac68e294a6b9f6cf624b097753a/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:01.025117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1e5a35354e4701990287503dcf2c0a, user_id=45e5eac68e294a6b9f6cf624b097753a, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:01.027779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:01.041828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1002/1999] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:01 2022] PUT /identity/v3/projects/ed1e5a35354e4701990287503dcf2c0a/users/45e5eac68e294a6b9f6cf624b097753a/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:01.054158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:01.055421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:01.055931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:01.088759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] MFA Rules not processed for user `45e5eac68e294a6b9f6cf624b097753a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:01.090038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:01.169070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 998/2000] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:01.177247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:01.179005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:01.254552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d9b0f-6ee8-4c6f-b687-5b4ad81b1958 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:01.254552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cd75f-979d-42df-8b47-5c93f7eb7ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:01.254552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cd75f-979d-42df-8b47-5c93f7eb7ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:01.254552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cd75f-979d-42df-8b47-5c93f7eb7ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:01.254912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cd75f-979d-42df-8b47-5c93f7eb7ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:01.303057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:01.325604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cd75f-979d-42df-8b47-5c93f7eb7ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:01.327151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cd75f-979d-42df-8b47-5c93f7eb7ea9 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:01.340587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1003/2001] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:03.911495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:03.981060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917241a-00b9-4689-b212-232bf17bd8f0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:03.985666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:03.986330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:03.986620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:03.986897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] PATH_INFO: `/v3/users/339a40a7ae1b4eb684c2952d5e7b6ff8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:03.988029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=339a40a7ae1b4eb684c2952d5e7b6ff8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:03.990251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:04.037381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339a40a7ae1b4eb684c2952d5e7b6ff8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.044365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339a40a7ae1b4eb684c2952d5e7b6ff8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.056077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339a40a7ae1b4eb684c2952d5e7b6ff8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.056683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339a40a7ae1b4eb684c2952d5e7b6ff8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.064062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339a40a7ae1b4eb684c2952d5e7b6ff8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.069805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 999/2002] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:03 2022] DELETE /identity/v3/users/339a40a7ae1b4eb684c2952d5e7b6ff8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:04.074371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:04.074837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:04.074947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:04.094358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:04.095549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:04.154594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1004/2003] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:04.164015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:04.165422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:04.171843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:04.223161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953d2d4-3318-40c4-9d25-a48107b9d81a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:04.226815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:04.227419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:04.227700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:04.227976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:04.236737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:04.238433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:04.247631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1000/2004] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:04.568311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:04.569713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:04.581686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee7ab-a79f-4abb-9711-a9659889c097 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:04.586409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:04.587082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:04.587363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:04.587643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] PATH_INFO: `/v3/projects/641f977a99cc483fbe08d5b6fdd7b46c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:04.588797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=641f977a99cc483fbe08d5b6fdd7b46c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:04.600842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:04.696427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641f977a99cc483fbe08d5b6fdd7b46c', 'request_id': 'req-6e62bc06-9441-45f0-affb-2ee4f76b80e3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.703794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641f977a99cc483fbe08d5b6fdd7b46c', 'request_id': 'req-6e62bc06-9441-45f0-affb-2ee4f76b80e3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.710508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641f977a99cc483fbe08d5b6fdd7b46c', 'request_id': 'req-6e62bc06-9441-45f0-affb-2ee4f76b80e3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.733458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] The token cache is being invalidate because project 641f977a99cc483fbe08d5b6fdd7b46c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:04.733857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.735915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.743695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1005/2005] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:04 2022] DELETE /identity/v3/projects/641f977a99cc483fbe08d5b6fdd7b46c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:04.744674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:04.759817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc736-93c6-4188-8406-087496a0be4f service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:04.847511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:04.848206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:04.848548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:04.848832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] PATH_INFO: `/v3/users/45e5eac68e294a6b9f6cf624b097753a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:04.850710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e5eac68e294a6b9f6cf624b097753a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:04.853204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:04.934379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e5eac68e294a6b9f6cf624b097753a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.947519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e5eac68e294a6b9f6cf624b097753a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.966133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e5eac68e294a6b9f6cf624b097753a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.967018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e5eac68e294a6b9f6cf624b097753a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.974264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-43452509-9b31-4f74-a235-61e91cd7447d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e5eac68e294a6b9f6cf624b097753a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:04.983859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1001/2006] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:04 2022] DELETE /identity/v3/users/45e5eac68e294a6b9f6cf624b097753a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:05.523216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:05.524700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:05.539470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62bc06-9441-45f0-affb-2ee4f76b80e3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:05.594966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:05.595597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:05.595879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:05.596158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] PATH_INFO: `/v3/projects/ed1e5a35354e4701990287503dcf2c0a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:05.597261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1e5a35354e4701990287503dcf2c0a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:05.599211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:05.601431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:05.648400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1e5a35354e4701990287503dcf2c0a', 'request_id': 'req-67302623-f260-4e09-881e-b7dcf7d42baf'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:05.654214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1e5a35354e4701990287503dcf2c0a', 'request_id': 'req-67302623-f260-4e09-881e-b7dcf7d42baf'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:05.660556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1e5a35354e4701990287503dcf2c0a', 'request_id': 'req-67302623-f260-4e09-881e-b7dcf7d42baf'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:05.672658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] The token cache is being invalidate because project ed1e5a35354e4701990287503dcf2c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:05.672952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:05.673606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-67302623-f260-4e09-881e-b7dcf7d42baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:05.675252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1006/2007] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:05 2022] DELETE /identity/v3/projects/ed1e5a35354e4701990287503dcf2c0a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:05.680293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341b601-350a-4878-b240-20d5813e47bc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:05.680585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341b601-350a-4878-b240-20d5813e47bc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:05.680863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341b601-350a-4878-b240-20d5813e47bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:05.702484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8341b601-350a-4878-b240-20d5813e47bc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:05.703898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8341b601-350a-4878-b240-20d5813e47bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:05.724097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:05.764807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1002/2008] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:05.773113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:05.773427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:05.773710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:05.793890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:05.795239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:05.804139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1007/2009] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:05.808415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341b601-350a-4878-b240-20d5813e47bc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:05.817854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341b601-350a-4878-b240-20d5813e47bc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:05.822746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:05.823366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:05.823647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:05.823932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:05.824683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:05.825032 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:05.826702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:05.838946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1003/2010] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:05.843544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:05.856741 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31fd5a-1d37-408a-9f11-20a49bbcac4a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:05.864048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:05.865681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:05.865681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:05.865681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:05.866321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:05.868321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:05.907870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1008/2011] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:05 2022] POST /identity/v3/projects => generated 397 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:05.911928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:05.936120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc01c-6b49-4462-a7e2-a4d05f437de2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:05.999248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:05.999864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.000166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.000444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.001086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:06.003030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:06.025278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1004/2012] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:05 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:06.029483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:06.041178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3050b-e971-4451-8a6e-6598d406a8a7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:06.046327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:06.047021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.047309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.047667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.048356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:06.050038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:06.059760 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1009/2013] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:06.063803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:06.073893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09153cf-92a4-4651-a987-34df92c0bc0f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:06.079608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a1a2f-b1f6-45ae-9dd0-a920caa818c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:06.080218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a1a2f-b1f6-45ae-9dd0-a920caa818c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.080511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a1a2f-b1f6-45ae-9dd0-a920caa818c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.080791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a1a2f-b1f6-45ae-9dd0-a920caa818c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.081512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a1a2f-b1f6-45ae-9dd0-a920caa818c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:06.083292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a1a2f-b1f6-45ae-9dd0-a920caa818c9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:06.094731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1005/2014] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:06.099730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:06.110371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2288a-f463-46b6-aa54-e6a479a12509 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:06.115515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:06.116258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.116518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.116819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] PATH_INFO: `/v3/projects/b05ef9356dcb4571bbeb93ef092ece90/users/4d11b16c46874e859e3028ec30b08e7a/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.146200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05ef9356dcb4571bbeb93ef092ece90, user_id=4d11b16c46874e859e3028ec30b08e7a, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:06.149342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:06.164220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1010/2015] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:06 2022] PUT /identity/v3/projects/b05ef9356dcb4571bbeb93ef092ece90/users/4d11b16c46874e859e3028ec30b08e7a/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:06.175110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.175401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.175681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.207972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] MFA Rules not processed for user `4d11b16c46874e859e3028ec30b08e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:06.209216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:06.302699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1006/2016] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:06.312092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:06.314853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:06.388161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d714da-0dac-4395-bc8d-78e8da491f13 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:06.392885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63afa29-ad27-4fc0-81ab-f07f197a78db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:06.393507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63afa29-ad27-4fc0-81ab-f07f197a78db service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.393789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63afa29-ad27-4fc0-81ab-f07f197a78db service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.394123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63afa29-ad27-4fc0-81ab-f07f197a78db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.403772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63afa29-ad27-4fc0-81ab-f07f197a78db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:06.407010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63afa29-ad27-4fc0-81ab-f07f197a78db service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:06.417963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1011/2017] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:06.583591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:06.585065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:06.612762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:06.660144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb228e-fc2e-4243-bfc9-3b5287b0b1e0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:06.664631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:06.665297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.665623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.665859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] PATH_INFO: `/v3/users/bba4ac3d238343c78bf23bbd76258a39` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.667003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba4ac3d238343c78bf23bbd76258a39)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:06.668959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:06.721857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba4ac3d238343c78bf23bbd76258a39'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:06.729855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba4ac3d238343c78bf23bbd76258a39'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:06.742976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba4ac3d238343c78bf23bbd76258a39'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:06.743628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba4ac3d238343c78bf23bbd76258a39'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:06.751601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba4ac3d238343c78bf23bbd76258a39'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:06.758661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1007/2018] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:06 2022] DELETE /identity/v3/users/bba4ac3d238343c78bf23bbd76258a39 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:06.763216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.763674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.763785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.782856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:06.784061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:06.811350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:06.849628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1012/2019] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:06.861568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:06.863816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:06.963963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b9b9-f8e9-4d49-9e68-f1e3ea9d1096 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:06.970488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:06.971253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:06.971591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:06.972019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:06.998295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:07.002179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:07.028709 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1008/2020] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:07.821607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:07.823308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:07.835960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fffe68-247b-4d80-a240-451f22e12d3a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:07.840912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:07.841545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:07.841825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:07.842132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] PATH_INFO: `/v3/projects/1a59516a57b54cb8a6b2b50b500f09e7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:07.843307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a59516a57b54cb8a6b2b50b500f09e7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:07.845281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:07.897682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a59516a57b54cb8a6b2b50b500f09e7', 'request_id': 'req-70374809-82f6-4b9c-92f5-698072998b58'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:07.904978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a59516a57b54cb8a6b2b50b500f09e7', 'request_id': 'req-70374809-82f6-4b9c-92f5-698072998b58'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:07.916199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a59516a57b54cb8a6b2b50b500f09e7', 'request_id': 'req-70374809-82f6-4b9c-92f5-698072998b58'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:07.934512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] The token cache is being invalidate because project 1a59516a57b54cb8a6b2b50b500f09e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:07.934960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:07.935680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:07.937211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1013/2021] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:07 2022] DELETE /identity/v3/projects/1a59516a57b54cb8a6b2b50b500f09e7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:07.942647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:07.955719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367f1b8-6672-43ff-8cd8-56c129deca67 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:08.009012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:08.027631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:08.028267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:08.028546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:08.028814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] PATH_INFO: `/v3/users/b9f819c39f22473fbdd1b08c71cab3d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:08.029985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f819c39f22473fbdd1b08c71cab3d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:08.031920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:08.087627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f819c39f22473fbdd1b08c71cab3d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:08.099766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f819c39f22473fbdd1b08c71cab3d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:08.116001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f819c39f22473fbdd1b08c71cab3d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:08.116639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f819c39f22473fbdd1b08c71cab3d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:08.125736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1fde6f27-bee2-4f38-8373-63bac102813c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f819c39f22473fbdd1b08c71cab3d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:08.145970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1009/2022] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:07 2022] DELETE /identity/v3/users/b9f819c39f22473fbdd1b08c71cab3d4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:09.005674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.007738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:09.023696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70374809-82f6-4b9c-92f5-698072998b58 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.114761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391dffa-1991-42da-8621-96138543844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.115734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391dffa-1991-42da-8621-96138543844c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.116181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391dffa-1991-42da-8621-96138543844c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.116515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391dffa-1991-42da-8621-96138543844c admin admin] PATH_INFO: `/v3/projects/4a986e884d184b3581ac1fd325d1573f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.117665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391dffa-1991-42da-8621-96138543844c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a986e884d184b3581ac1fd325d1573f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.122317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391dffa-1991-42da-8621-96138543844c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.181651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5391dffa-1991-42da-8621-96138543844c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a986e884d184b3581ac1fd325d1573f', 'request_id': 'req-5391dffa-1991-42da-8621-96138543844c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:09.186717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:09.187976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5391dffa-1991-42da-8621-96138543844c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a986e884d184b3581ac1fd325d1573f', 'request_id': 'req-5391dffa-1991-42da-8621-96138543844c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:09.196073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5391dffa-1991-42da-8621-96138543844c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a986e884d184b3581ac1fd325d1573f', 'request_id': 'req-5391dffa-1991-42da-8621-96138543844c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:09.210132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5391dffa-1991-42da-8621-96138543844c admin admin] The token cache is being invalidate because project 4a986e884d184b3581ac1fd325d1573f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:09.210425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5391dffa-1991-42da-8621-96138543844c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:09.211074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5391dffa-1991-42da-8621-96138543844c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:09.212568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1014/2023] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:09 2022] DELETE /identity/v3/projects/4a986e884d184b3581ac1fd325d1573f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:09.217439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.217940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.218034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.238188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:09.239677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:09.305190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1010/2024] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:09.313313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.313688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.314028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.339341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:09.340533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:09.349850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1015/2025] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:09.354331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.368765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7319db-9f44-465b-9477-4d54c128e2c6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.373654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.374397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.374741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.375534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.376251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:09.376778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.379129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.395317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1011/2026] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:09.396838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.413024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82352a01-05c3-4aec-b83a-41349b78b497 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.425356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.426095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.426449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.426795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.427550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.433880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.466801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.467255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1016/2027] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:09.478529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82773d15-c32f-4acb-aaa9-ab9264dc6593 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.528316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:09.540677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.541358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.541708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.542084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.542716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.544640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.566648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1012/2028] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:09.570593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.581101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67c709-a59c-4388-a3ba-58e1a71561e4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.585706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.586341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.586630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.586892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.587532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.589116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.602824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1017/2029] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:09.603963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.614314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93e999-34d9-48d9-b246-436a4ab9e4a7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.619764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f2839-6cfb-4f0e-ad50-8bec88a340a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.619764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f2839-6cfb-4f0e-ad50-8bec88a340a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.619987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f2839-6cfb-4f0e-ad50-8bec88a340a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.620264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f2839-6cfb-4f0e-ad50-8bec88a340a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.620912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f2839-6cfb-4f0e-ad50-8bec88a340a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.622547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f2839-6cfb-4f0e-ad50-8bec88a340a1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.634210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1013/2030] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:09.638464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.649016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014f7a1-6797-4fad-a494-36ebc40ff729 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.653589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.654353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.654571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.654840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] PATH_INFO: `/v3/projects/e4d0e2406c484114b107e2ac94f13cdf/users/7fb3d25634ff4372884114bd019f35b5/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.673657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d0e2406c484114b107e2ac94f13cdf, user_id=7fb3d25634ff4372884114bd019f35b5, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.676410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.692316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1018/2031] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:09 2022] PUT /identity/v3/projects/e4d0e2406c484114b107e2ac94f13cdf/users/7fb3d25634ff4372884114bd019f35b5/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:09.702577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.702867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.703196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.732696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] MFA Rules not processed for user `7fb3d25634ff4372884114bd019f35b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:09.733894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:09.819105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1014/2032] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:09.823793 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.884857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62e230-3004-4c0c-8096-8449dbfbdc7d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.889840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.891020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.891300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.891571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.892190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.894120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:09.920138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1019/2033] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:09.924320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:09.935199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df43f7-9d96-4c63-a5af-8da4e0a32853 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:09.991192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:09.991797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:09.992079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:09.992358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:09.992979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:09.994936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:10.015972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1015/2034] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:09 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:10.020490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:10.031053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f26557-1528-4093-9743-0b68b4da953a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:10.035641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:10.036227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:10.036500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:10.036769 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:10.037429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:10.039065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:10.049334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1020/2035] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:10.053174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:10.063346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b57b3-cbdf-455d-99da-45b0679ce69b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:10.067916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:10.068547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:10.068828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:10.069101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] PATH_INFO: `/v3/projects/db588c5d8b0e425bb1ca3e079109e621/users/d1c8747e713a49bf9026b91e7210a32e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:10.083870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db588c5d8b0e425bb1ca3e079109e621, user_id=d1c8747e713a49bf9026b91e7210a32e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:10.086543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:10.099327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1016/2036] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:10 2022] PUT /identity/v3/projects/db588c5d8b0e425bb1ca3e079109e621/users/d1c8747e713a49bf9026b91e7210a32e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:10.103314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:10.113084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0156f-190c-483b-8962-087136f9daf3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:10.164218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:10.172083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e10e07-a1cf-4081-aad6-f836c48c6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:10.172641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e10e07-a1cf-4081-aad6-f836c48c6b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:10.172928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e10e07-a1cf-4081-aad6-f836c48c6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:10.173429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e10e07-a1cf-4081-aad6-f836c48c6b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:10.174566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e10e07-a1cf-4081-aad6-f836c48c6b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:10.177492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e10e07-a1cf-4081-aad6-f836c48c6b6a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:10.188349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1021/2037] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:10.192629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:10.202699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d17230-c760-44ef-b330-eb8688b55f2b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:10.207788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:10.208437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:10.208715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:10.208990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] PATH_INFO: `/v3/projects/db588c5d8b0e425bb1ca3e079109e621/users/d1c8747e713a49bf9026b91e7210a32e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:10.210996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db588c5d8b0e425bb1ca3e079109e621, user_id=d1c8747e713a49bf9026b91e7210a32e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:10.213615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:10.224454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1017/2038] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:10 2022] PUT /identity/v3/projects/db588c5d8b0e425bb1ca3e079109e621/users/d1c8747e713a49bf9026b91e7210a32e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:10.235604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:10.235894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:10.236169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:10.264440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] MFA Rules not processed for user `d1c8747e713a49bf9026b91e7210a32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:10.266263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:10.336209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1022/2039] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:10.346715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:10.348390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:10.362838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:10.410389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8962263-edb6-47a5-a9bc-8e3e61cdd760 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:10.413968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f532383-5585-4737-986a-fca81bd77200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:10.414565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f532383-5585-4737-986a-fca81bd77200 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:10.414846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f532383-5585-4737-986a-fca81bd77200 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:10.415116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f532383-5585-4737-986a-fca81bd77200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:10.424415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f532383-5585-4737-986a-fca81bd77200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:10.425766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f532383-5585-4737-986a-fca81bd77200 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:10.435064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1018/2040] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:11.883309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:11.961373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db157b-d43e-49e5-9cab-e67a8af2af9d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:11.966285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:11.966935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:11.967221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:11.967615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] PATH_INFO: `/v3/users/7fb3d25634ff4372884114bd019f35b5` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:11.969331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb3d25634ff4372884114bd019f35b5)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:11.971431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:12.036075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb3d25634ff4372884114bd019f35b5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.041071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb3d25634ff4372884114bd019f35b5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.057882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb3d25634ff4372884114bd019f35b5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.058710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb3d25634ff4372884114bd019f35b5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.064230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:12.066243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb3d25634ff4372884114bd019f35b5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.074535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1023/2041] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:11 2022] DELETE /identity/v3/users/7fb3d25634ff4372884114bd019f35b5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:12.083524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:12.084035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:12.084138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:12.107478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:12.108999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:12.173494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1019/2042] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:12.183772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:12.185624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:12.252842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75ae5-e72f-45b2-84f6-8d7804b67a03 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:12.256470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:12.257089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:12.257374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:12.257648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:12.269560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:12.271565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:12.282765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1024/2043] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:12.613083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:12.614888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:12.637629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a04cb-79b8-4455-b662-8d2db18a77cb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:12.647606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:12.648270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:12.648551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:12.649553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] PATH_INFO: `/v3/projects/e4d0e2406c484114b107e2ac94f13cdf` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:12.650763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d0e2406c484114b107e2ac94f13cdf)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:12.660363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:12.752148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d0e2406c484114b107e2ac94f13cdf', 'request_id': 'req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.759409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d0e2406c484114b107e2ac94f13cdf', 'request_id': 'req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.768862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d0e2406c484114b107e2ac94f13cdf', 'request_id': 'req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.786094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] The token cache is being invalidate because project e4d0e2406c484114b107e2ac94f13cdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:12.786400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.787131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e79bfcd3-56ff-4a2b-80c1-1533e1be5786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.790288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1020/2044] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:12 2022] DELETE /identity/v3/projects/e4d0e2406c484114b107e2ac94f13cdf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:12.794058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:12.804652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e0f6-c515-45f8-9d70-3dda77c73482 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:12.865364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:12.866050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:12.866328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:12.866598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] PATH_INFO: `/v3/users/d1c8747e713a49bf9026b91e7210a32e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:12.867749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c8747e713a49bf9026b91e7210a32e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:12.869705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:12.928417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c8747e713a49bf9026b91e7210a32e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.938195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c8747e713a49bf9026b91e7210a32e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.951024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c8747e713a49bf9026b91e7210a32e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.951873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c8747e713a49bf9026b91e7210a32e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.959273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c8747e713a49bf9026b91e7210a32e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:12.965220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1025/2045] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:12 2022] DELETE /identity/v3/users/d1c8747e713a49bf9026b91e7210a32e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:13.245944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:13.246447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:13.246526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:13.265810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:13.267041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:13.301353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:13.327200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1021/2046] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:13.373207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:13.374659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:13.399682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:13.478384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fefe8b-998d-44d0-9b05-1488956cb059 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:13.485771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06013ec2-cd88-4cca-8f5e-fe0325504529 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:13.486454 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06013ec2-cd88-4cca-8f5e-fe0325504529 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:13.486751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06013ec2-cd88-4cca-8f5e-fe0325504529 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:13.487066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06013ec2-cd88-4cca-8f5e-fe0325504529 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:13.499433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06013ec2-cd88-4cca-8f5e-fe0325504529 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:13.501777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06013ec2-cd88-4cca-8f5e-fe0325504529 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:13.514316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1026/2047] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 15:01:13.533471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:13.535821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:13.622853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329245b-cd85-4c6d-80fe-1ff267cd0e35 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:13.627769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:13.628479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:13.628764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:13.629204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] PATH_INFO: `/v3/projects/db588c5d8b0e425bb1ca3e079109e621` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:13.630393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=db588c5d8b0e425bb1ca3e079109e621)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:13.632313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:13.698228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db588c5d8b0e425bb1ca3e079109e621', 'request_id': 'req-d687d570-b8e4-4549-9614-9e25f0b5cecb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:13.706819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db588c5d8b0e425bb1ca3e079109e621', 'request_id': 'req-d687d570-b8e4-4549-9614-9e25f0b5cecb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:13.715839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db588c5d8b0e425bb1ca3e079109e621', 'request_id': 'req-d687d570-b8e4-4549-9614-9e25f0b5cecb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:13.743183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] The token cache is being invalidate because project db588c5d8b0e425bb1ca3e079109e621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:13.743495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:13.744582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d687d570-b8e4-4549-9614-9e25f0b5cecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:13.746834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1022/2048] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:13 2022] DELETE /identity/v3/projects/db588c5d8b0e425bb1ca3e079109e621 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:13.754186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:13.754815 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:13.755162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:13.791787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:13.795878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:13.883066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1027/2049] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:13.892251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:13.892542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:13.892828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:13.929850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:13.933428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:13.969583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1023/2050] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:13.974239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:13.986112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc4112-aa4f-4181-9fed-6b648fbd273d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:13.991390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:13.992253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:13.993448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:13.994443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:13.995242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:13.995616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:13.997459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.011696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1028/2051] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:14.016581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.025757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993fbfe-473e-42ef-9fdb-198313b20b31 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.030902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.031618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.031966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.032308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.033022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.035026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.070406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1024/2052] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:14 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:14.074396 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.087962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0075ec-02cc-4a51-ab85-6cd856116dbc admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.169511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.170319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.170579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.170830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.171543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.174470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.201703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1029/2053] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:14 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:14.207604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.217735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035712f8-e674-410f-a721-1f519fb1f9ac admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.222906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.223632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.223942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.224389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.225106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.226938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.239513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1025/2054] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:14.243798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.255202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0af73-5201-40ab-9fc1-5af5eac758e3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.261241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941129e-1e62-4102-8c95-7ee7fdcbf685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.262033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941129e-1e62-4102-8c95-7ee7fdcbf685 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.262312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941129e-1e62-4102-8c95-7ee7fdcbf685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.262615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941129e-1e62-4102-8c95-7ee7fdcbf685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.263263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941129e-1e62-4102-8c95-7ee7fdcbf685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.265270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941129e-1e62-4102-8c95-7ee7fdcbf685 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.280552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1030/2055] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:14.285784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.298735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad50d9f-4608-48b4-a3ce-4a9be2808720 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.304871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c948f-1062-4347-8567-1a8e1e463eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.305536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c948f-1062-4347-8567-1a8e1e463eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.306023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c948f-1062-4347-8567-1a8e1e463eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.306302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c948f-1062-4347-8567-1a8e1e463eac admin admin] PATH_INFO: `/v3/projects/2ff5e0bb110f4ab29c13eb21f3bbc4f1/users/2b5747527ba948de96d95fb934b59e21/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.323000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c948f-1062-4347-8567-1a8e1e463eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff5e0bb110f4ab29c13eb21f3bbc4f1, user_id=2b5747527ba948de96d95fb934b59e21, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.326651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c948f-1062-4347-8567-1a8e1e463eac admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.343846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1026/2056] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:14 2022] PUT /identity/v3/projects/2ff5e0bb110f4ab29c13eb21f3bbc4f1/users/2b5747527ba948de96d95fb934b59e21/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:14.354899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.355278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.355873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.382121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.382622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.383220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.387742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:14.391522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] MFA Rules not processed for user `2b5747527ba948de96d95fb934b59e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:14.394413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:14.414844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:14.416269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:14.423872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:14.501042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1027/2057] 10.222.0.61 () {58 vars in 967 bytes} [Sat Sep 24 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:14.507715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1031/2058] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:14.514756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.515298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.516452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:14.613798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b0ff2-d0be-4cd0-8f69-eb0cdd8b6808 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.614213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d2687-332d-4771-b5f9-51ac27e3abcd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.619731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.620520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.620944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.621360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.627728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.628351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.628641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.629523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.631179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.631730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.633681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.634288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.644652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1032/2059] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:01:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:01:14.727884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1028/2060] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:14 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:14.730084 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.744284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a462b6-1871-4c44-b058-a89fb593baa1 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.840623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.841261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.841543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.841832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.842512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.845009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.873186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1033/2061] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:14 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:14.874613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.885141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82fcd-2aed-40a5-9aeb-e918d2e9c7e6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.891347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.891954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.892238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.892514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.893163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.894809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.909981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1029/2062] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:14.911900 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.923130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819a352-c3a7-4bda-8b55-c262dd4f5b11 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:14.932444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:14.932444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:14.932444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:14.932444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] PATH_INFO: `/v3/projects/ca83afe67f424cf6a794e592eff9ca7c/users/ecc6773ab87249bbaffaaa49d8baf4d7/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:14.950242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca83afe67f424cf6a794e592eff9ca7c, user_id=ecc6773ab87249bbaffaaa49d8baf4d7, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:14.953578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:14.965478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:14.982820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1034/2063] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:14 2022] PUT /identity/v3/projects/ca83afe67f424cf6a794e592eff9ca7c/users/ecc6773ab87249bbaffaaa49d8baf4d7/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:14.983406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:14.996324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5a57e-bcec-40e6-bc7e-0ef1683e7fba admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.008270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:15.010946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.123189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8d28a-316a-4b0e-be73-f3ddabe6eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.123775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8d28a-316a-4b0e-be73-f3ddabe6eceb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.124061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8d28a-316a-4b0e-be73-f3ddabe6eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.124331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8d28a-316a-4b0e-be73-f3ddabe6eceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.130330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8d28a-316a-4b0e-be73-f3ddabe6eceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:15.132639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8d28a-316a-4b0e-be73-f3ddabe6eceb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:15.139177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d3d0a-69d5-4d59-abd6-45adaafe5eda admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.145130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.145790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.146099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.146376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] PATH_INFO: `/v3/users/4d11b16c46874e859e3028ec30b08e7a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.149741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1030/2064] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:15.149741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.149741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.149741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.150570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d11b16c46874e859e3028ec30b08e7a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:15.152532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:15.170529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:15.172296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.195224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:15.206834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d11b16c46874e859e3028ec30b08e7a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:15.217983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d11b16c46874e859e3028ec30b08e7a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:15.232291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d11b16c46874e859e3028ec30b08e7a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:15.233022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d11b16c46874e859e3028ec30b08e7a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:15.245634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1031/2065] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:15.246781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:15.248203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.249521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d11b16c46874e859e3028ec30b08e7a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:15.263932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1035/2066] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:15 2022] DELETE /identity/v3/users/4d11b16c46874e859e3028ec30b08e7a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:15.263932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:15.266816 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.272047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7879f-161d-4da5-9ec6-666b0651d69e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.354376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f776-a05f-44cb-a28c-f8ecead53beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.355296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f776-a05f-44cb-a28c-f8ecead53beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.355765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f776-a05f-44cb-a28c-f8ecead53beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.356169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f776-a05f-44cb-a28c-f8ecead53beb admin admin] PATH_INFO: `/v3/projects/ca83afe67f424cf6a794e592eff9ca7c/users/ecc6773ab87249bbaffaaa49d8baf4d7/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.358401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43f776-a05f-44cb-a28c-f8ecead53beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca83afe67f424cf6a794e592eff9ca7c, user_id=ecc6773ab87249bbaffaaa49d8baf4d7, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:15.361971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43f776-a05f-44cb-a28c-f8ecead53beb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:15.374069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1032/2067] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:15 2022] PUT /identity/v3/projects/ca83afe67f424cf6a794e592eff9ca7c/users/ecc6773ab87249bbaffaaa49d8baf4d7/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:15.375520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60dc10-b55f-4415-b1be-4c927fd889c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.376056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60dc10-b55f-4415-b1be-4c927fd889c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.376433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60dc10-b55f-4415-b1be-4c927fd889c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.390501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea384afd-3e0e-4510-ae10-7279d1cb5ebe admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.400257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6b60dc10-b55f-4415-b1be-4c927fd889c4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:15.401999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b60dc10-b55f-4415-b1be-4c927fd889c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.431555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:15.516029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1033/2068] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:15.521053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.521725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.522134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.541066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.541702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.542012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.542287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.570229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] MFA Rules not processed for user `ecc6773ab87249bbaffaaa49d8baf4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:15.571617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.636363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:15.647509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:15.649217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:15.658638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1036/2069] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:01:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:01:15.659576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:15.662232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.696203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1034/2070] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:15.712459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:15.715295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:15.755662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a05e76-ee43-49d8-b1c7-81a37fb78004 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.761859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.763143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.763542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.763882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.782013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:15.783529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:15.794693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1037/2071] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:15.829767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942c1a-4725-46b2-bf71-ca77a1eab8cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.836230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84fff9-ef75-4470-abb5-8008a483b91c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.836898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84fff9-ef75-4470-abb5-8008a483b91c service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.837372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84fff9-ef75-4470-abb5-8008a483b91c service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.837757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84fff9-ef75-4470-abb5-8008a483b91c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.847999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84fff9-ef75-4470-abb5-8008a483b91c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:15.849641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84fff9-ef75-4470-abb5-8008a483b91c service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:15.871065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1035/2072] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:15.927360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:15.939853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb45902-6f15-428c-9ac3-8c5853bca742 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:15.944168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:15.944836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:15.945227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:15.945586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:15.947449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:16.030595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:16.032249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:16.043491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1038/2073] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:16.254539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:16.254842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:16.255253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:16.289397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:16.291304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:16.309545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1036/2074] 10.222.0.61 () {58 vars in 967 bytes} [Sat Sep 24 15:01:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:16.333748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:16.339908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:16.348797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76d3e8-93b1-4fe9-988f-26d81df8eef5 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:16.356751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:16.357834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:16.358242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:16.359352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:16.372544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:16.374194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:16.383878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1039/2075] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:01:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:01:16.619354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:16.622518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:16.646484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2042da5-6b1b-4ee1-baa7-aaa15aab6781 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:16.653434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:16.654247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:16.654607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:16.654789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] PATH_INFO: `/v3/projects/b05ef9356dcb4571bbeb93ef092ece90` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:16.656281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05ef9356dcb4571bbeb93ef092ece90)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:16.658535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:16.831896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05ef9356dcb4571bbeb93ef092ece90', 'request_id': 'req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:16.847033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05ef9356dcb4571bbeb93ef092ece90', 'request_id': 'req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:16.867507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05ef9356dcb4571bbeb93ef092ece90', 'request_id': 'req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:16.903726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] The token cache is being invalidate because project b05ef9356dcb4571bbeb93ef092ece90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:16.904025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:16.904693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:16.907183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1037/2076] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:16 2022] DELETE /identity/v3/projects/b05ef9356dcb4571bbeb93ef092ece90 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:17.210912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:17.212944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:17.228366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b7dc5-d923-440f-bddf-6f8c8ad206b8 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:17.286413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:17.287045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:17.287326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:17.287621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] PATH_INFO: `/v3/users/5b63ee5b9ac44eb39617b291ac869db7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:17.288786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b63ee5b9ac44eb39617b291ac869db7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:17.290754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:17.357710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b63ee5b9ac44eb39617b291ac869db7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.367883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b63ee5b9ac44eb39617b291ac869db7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.386640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b63ee5b9ac44eb39617b291ac869db7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.387413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b63ee5b9ac44eb39617b291ac869db7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.398341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-959f9969-92b0-4c50-9f13-b77150d201f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b63ee5b9ac44eb39617b291ac869db7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.405307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1040/2077] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:17 2022] DELETE /identity/v3/users/5b63ee5b9ac44eb39617b291ac869db7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:17.608169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:17.618000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:17.647964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa71c4-25c7-4f07-a74f-d8b76ef8f181 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:17.735370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:17.736576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:17.736877 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:17.737721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] PATH_INFO: `/v3/projects/5a28fd8d62a54976b961c31176fed822` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:17.740033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a28fd8d62a54976b961c31176fed822)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:17.744567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:17.760204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:17.815917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a28fd8d62a54976b961c31176fed822', 'request_id': 'req-108d9264-4e49-4431-9953-0142d94dabd6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.844790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a28fd8d62a54976b961c31176fed822', 'request_id': 'req-108d9264-4e49-4431-9953-0142d94dabd6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.855397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a28fd8d62a54976b961c31176fed822', 'request_id': 'req-108d9264-4e49-4431-9953-0142d94dabd6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.874315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] The token cache is being invalidate because project 5a28fd8d62a54976b961c31176fed822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:17.875156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.875878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-108d9264-4e49-4431-9953-0142d94dabd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:17.892067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1038/2078] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:17 2022] DELETE /identity/v3/projects/5a28fd8d62a54976b961c31176fed822 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:17.896733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:17.897242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:17.897952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:17.947776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:17.949864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:18.033564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1041/2079] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:18.041470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.041892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.042330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.071950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:18.073186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:18.087358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1039/2080] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:18.092010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.105664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c736b7-fd2f-4f5c-85f6-4a9317e1c6e3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.122278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.122278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.122278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.122278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.122678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:18.122678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.122678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.122678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:18.139128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1042/2081] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:18.140800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.153678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9c18-3a24-46f3-ac44-70e5dd535c9b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.159330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.160095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.160496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.160879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.161759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.164065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.195096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1040/2082] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:18 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:18.199410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.209871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbaafd-12c0-416f-a094-d8e56f6738ab admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.271744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.272341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.272615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.272894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.273525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.275661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.300865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1043/2083] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:18 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:18.304941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.317803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47953d2e-13e5-4a46-aa96-055ca365b088 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.325242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.326006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.326365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.326749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.327580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.330132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.341059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1041/2084] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:18.345115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.355526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58d56-cd3e-4997-9967-bcc16c6ace85 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.360428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2c83b-9f5a-4bd6-aa91-77d68b8faf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.361083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2c83b-9f5a-4bd6-aa91-77d68b8faf9e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.361394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2c83b-9f5a-4bd6-aa91-77d68b8faf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.361668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2c83b-9f5a-4bd6-aa91-77d68b8faf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.362340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2c83b-9f5a-4bd6-aa91-77d68b8faf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.364236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2c83b-9f5a-4bd6-aa91-77d68b8faf9e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.375928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1044/2085] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:18.380344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.392297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582a68e-a800-484d-bb11-9a8e3a15259b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.396884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.397529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.397813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.398127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] PATH_INFO: `/v3/projects/a638b5af9be44eaa852493199e8c880f/users/12a7df242e2b4e898d683d42a3836f65/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.412355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a638b5af9be44eaa852493199e8c880f, user_id=12a7df242e2b4e898d683d42a3836f65, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.415141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.427077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1042/2086] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:18 2022] PUT /identity/v3/projects/a638b5af9be44eaa852493199e8c880f/users/12a7df242e2b4e898d683d42a3836f65/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:18.438038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.438329 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.438600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.466259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] MFA Rules not processed for user `12a7df242e2b4e898d683d42a3836f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:18.467457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:18.529429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1045/2087] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:18.540324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.542045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:18.613036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b311-8f09-48a8-88aa-6e7d791a12a6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.618828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.619430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.619712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.620026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.628640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.630266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.638906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1043/2088] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:18.871830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.941545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1adf4-6f2c-434b-8189-88d5f9fe257a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.947630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.948271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.948551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.948825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] PATH_INFO: `/v3/users/12a7df242e2b4e898d683d42a3836f65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.950353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a7df242e2b4e898d683d42a3836f65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.952285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:18.959120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:18.961160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:18.962904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:18.980492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135cab-0c2a-4c7b-913b-6b2630744ab5 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:18.986296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:18.986926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:18.987233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:18.987506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] PATH_INFO: `/v3/users/d80d5192d7224516bdff0af8c99f5cea` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:18.988931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80d5192d7224516bdff0af8c99f5cea)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:18.991102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:19.006851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:19.007597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a7df242e2b4e898d683d42a3836f65'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.018881 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a7df242e2b4e898d683d42a3836f65'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.045392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a7df242e2b4e898d683d42a3836f65'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.046165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a7df242e2b4e898d683d42a3836f65'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.050636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80d5192d7224516bdff0af8c99f5cea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.055199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e30e012-668a-44f4-830f-5025a9a6abc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a7df242e2b4e898d683d42a3836f65'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.060667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80d5192d7224516bdff0af8c99f5cea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.063389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1046/2089] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:18 2022] DELETE /identity/v3/users/12a7df242e2b4e898d683d42a3836f65 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:19.067824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.068114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.068809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.080910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80d5192d7224516bdff0af8c99f5cea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.081775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80d5192d7224516bdff0af8c99f5cea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.090256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e3ee9eb-81a7-4c34-8d25-afcda3e9463d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80d5192d7224516bdff0af8c99f5cea'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.091899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:19.098405 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1044/2090] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:18 2022] DELETE /identity/v3/users/d80d5192d7224516bdff0af8c99f5cea => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:19.102876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.103889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.103889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.104713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.143343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:19.144881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.230104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1047/2091] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:19.231721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1045/2092] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:19.244120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:19.245581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.257045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:19.258386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.330883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0ad31-bebd-4430-8209-e85a9679dd0e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:19.334972 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc707e2f-2af9-467f-b2d3-cf819b73492f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:19.336369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:19.337058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.337398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.337674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.340815 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:19.341995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.342460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.342912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.353059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:19.355304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:19.357286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:19.359601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:19.373450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1046/2093] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:01:19.375226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1048/2094] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:19.742430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:19.744054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.772180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa3ee1-1e88-45bf-a017-0f165076cf65 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:19.779982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:19.780632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.780941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.781220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] PATH_INFO: `/v3/projects/eddca7f4673e49dc93a6e1c15e78c521` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.782512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddca7f4673e49dc93a6e1c15e78c521)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:19.785320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:19.843062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:19.862892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:19.864189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddca7f4673e49dc93a6e1c15e78c521', 'request_id': 'req-84799ee1-f713-4e6a-be3a-ea946d7ccf63'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.865847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.878184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddca7f4673e49dc93a6e1c15e78c521', 'request_id': 'req-84799ee1-f713-4e6a-be3a-ea946d7ccf63'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.890843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddca7f4673e49dc93a6e1c15e78c521', 'request_id': 'req-84799ee1-f713-4e6a-be3a-ea946d7ccf63'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.894203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cfaec-9544-4ecb-a030-8201f88be4e5 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:19.927277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:19.934380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] The token cache is being invalidate because project eddca7f4673e49dc93a6e1c15e78c521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:19.934678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.935447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-84799ee1-f713-4e6a-be3a-ea946d7ccf63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:19.938035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1047/2095] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:19 2022] DELETE /identity/v3/projects/eddca7f4673e49dc93a6e1c15e78c521 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:19.943460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be317366-6e79-40f3-8735-935a37017623 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.944871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be317366-6e79-40f3-8735-935a37017623 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.945549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be317366-6e79-40f3-8735-935a37017623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.972223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-be317366-6e79-40f3-8735-935a37017623 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:19.975360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-be317366-6e79-40f3-8735-935a37017623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:19.987195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:19.988029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:19.988568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:19.988843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] PATH_INFO: `/v3/projects/a638b5af9be44eaa852493199e8c880f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:19.990019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a638b5af9be44eaa852493199e8c880f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:19.992306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.050225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a638b5af9be44eaa852493199e8c880f', 'request_id': 'req-b5c5eed2-b694-4a74-ac7a-80f86604d507'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.063114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1048/2096] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:20.069469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a638b5af9be44eaa852493199e8c880f', 'request_id': 'req-b5c5eed2-b694-4a74-ac7a-80f86604d507'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.069554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be317366-6e79-40f3-8735-935a37017623 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.069554 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-be317366-6e79-40f3-8735-935a37017623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.074985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a638b5af9be44eaa852493199e8c880f', 'request_id': 'req-b5c5eed2-b694-4a74-ac7a-80f86604d507'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.082457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be317366-6e79-40f3-8735-935a37017623 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.089967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] The token cache is being invalidate because project a638b5af9be44eaa852493199e8c880f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:20.090265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.090907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b5c5eed2-b694-4a74-ac7a-80f86604d507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.092436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1049/2097] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:19 2022] DELETE /identity/v3/projects/a638b5af9be44eaa852493199e8c880f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:20.096909 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbadf3-d8e1-4e82-8ff2-310885b17a42 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.097200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbadf3-d8e1-4e82-8ff2-310885b17a42 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.097478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbadf3-d8e1-4e82-8ff2-310885b17a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.116820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4bdbadf3-d8e1-4e82-8ff2-310885b17a42 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:20.118059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4bdbadf3-d8e1-4e82-8ff2-310885b17a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.143417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.144048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.144326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.144601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] PATH_INFO: `/v3/users/2b5747527ba948de96d95fb934b59e21` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.145751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5747527ba948de96d95fb934b59e21)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.147886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.182993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1050/2098] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:20.184219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.184554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.184783 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.203187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5747527ba948de96d95fb934b59e21'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.206178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:20.207368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.210477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5747527ba948de96d95fb934b59e21'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.215667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1051/2099] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:20.216440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.216992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:20.224843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5747527ba948de96d95fb934b59e21'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.226062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5747527ba948de96d95fb934b59e21'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.229259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaa4b2-3c44-4a92-b584-13d255d609e4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.234549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d978fae0-3b89-4542-82df-5e1f0203f91f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5747527ba948de96d95fb934b59e21'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.234673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.235184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.235464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.235854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.236624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:20.236735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.238916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766770a0-2756-4fc1-85ab-78efd1dcecc1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.240842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1049/2100] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:20 2022] DELETE /identity/v3/users/2b5747527ba948de96d95fb934b59e21 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:20.242070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.242358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.242635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.251292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1052/2101] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:20.252648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.252936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.253213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.262873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:20.264003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.271350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:20.272512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.305857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:20.326563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1050/2102] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:20.329809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.329809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.338587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1053/2103] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:20.339480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.339979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0388f-5d87-4f1e-9943-6fc422352705 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.346551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.347344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.347463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.347838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.348512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.349150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73504463-e445-461c-b280-4b4a4bb41a5e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.350543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.353722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.354349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.354625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.354898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.355533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:20.355831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.357414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.370415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1054/2104] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:20.371517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.376698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.387981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1051/2105] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:20.388779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.449101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4629a-2144-4671-87fa-75de4078c82c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.466518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ee86-a96f-4016-97ec-afa86bdb1de6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.471265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.471894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.472175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.472456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.473118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.475075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.506929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1052/2106] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:20.507765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.510380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.511161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.512048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.512048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.522474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.523925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.547312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1055/2107] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:20.548384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.562682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a46e0b-4e89-4ebf-8cb5-aa38a9534026 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.568579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:20.596851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fed2b-804c-4b78-b79c-9e9da55bc53c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.604332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.604992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.605508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.605793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.606528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.608501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.637252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1053/2108] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:20.638805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.648742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2eedf-c979-4d2f-b0d9-87e6d1624ab9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.653662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.654366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.654726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.655074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.656337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.658140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.659750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.660353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.660655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.660954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.661554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.663605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.670702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1054/2109] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:20.675747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.688456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca32300-a021-42cf-863c-4516613a2ca8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.690865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1056/2110] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/users => generated 302 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:20.693757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.694337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.694674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.694901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.695455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.695596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.697185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.708740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2033f-e213-45a6-adf7-2ad66dd51ae6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.708879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1055/2111] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:20.715205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.725253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.726266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.726702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.727122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.727926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.729076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6965b08-6732-46c7-b126-ff0987e954a0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.730297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.735736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4a456-59b1-4bd0-8d8b-6b6b6fe63be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.736683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a456-59b1-4bd0-8d8b-6b6b6fe63be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.736971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a456-59b1-4bd0-8d8b-6b6b6fe63be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.737587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4a456-59b1-4bd0-8d8b-6b6b6fe63be7 admin admin] PATH_INFO: `/v3/projects/5e00eaec5c3a4b53b9e39346e792da01/users/482233dae68d487ca9b2ff6c69684663/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.747414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1057/2112] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:20.748400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.754211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a456-59b1-4bd0-8d8b-6b6b6fe63be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e00eaec5c3a4b53b9e39346e792da01, user_id=482233dae68d487ca9b2ff6c69684663, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.754504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:20.757321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4a456-59b1-4bd0-8d8b-6b6b6fe63be7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.762374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b721cf-1565-42cb-b803-1a7429cacc4e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.770377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.771043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.771325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.771946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.772280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.773949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1056/2113] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:20 2022] PUT /identity/v3/projects/5e00eaec5c3a4b53b9e39346e792da01/users/482233dae68d487ca9b2ff6c69684663/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:20.777381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.778897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.779334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.779754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.790279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1058/2114] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:20.794950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.802906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:20.804132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.807376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae609-87e3-454c-8420-253bee4339ab admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.868553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1057/2115] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:20.870861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.875074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.884419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.885204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.885556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.885932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] PATH_INFO: `/v3/projects/108f2369986c4d6eb65e285fa4e7e823/users/8625d0b6889146dca447748ab1ae86b8/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.887588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc2675-9d9f-4b9f-9cce-b4b5a07effba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.892881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:20.893513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.893795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.894101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] PATH_INFO: `/v3/projects/2ff5e0bb110f4ab29c13eb21f3bbc4f1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.895399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff5e0bb110f4ab29c13eb21f3bbc4f1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.897327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.906673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=108f2369986c4d6eb65e285fa4e7e823, user_id=8625d0b6889146dca447748ab1ae86b8, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:20.909852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:20.923245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1059/2116] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:20 2022] PUT /identity/v3/projects/108f2369986c4d6eb65e285fa4e7e823/users/8625d0b6889146dca447748ab1ae86b8/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:20.933691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:20.935447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:20.947955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff5e0bb110f4ab29c13eb21f3bbc4f1', 'request_id': 'req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.954277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff5e0bb110f4ab29c13eb21f3bbc4f1', 'request_id': 'req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.960426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff5e0bb110f4ab29c13eb21f3bbc4f1', 'request_id': 'req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.973959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] The token cache is being invalidate because project 2ff5e0bb110f4ab29c13eb21f3bbc4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:20.974558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.974917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0338955d-7c2a-4a14-b3c9-9237ec1cfb4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:20.979711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1058/2117] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:20 2022] DELETE /identity/v3/projects/2ff5e0bb110f4ab29c13eb21f3bbc4f1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:20.979711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:20.979711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:20.979711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:20.995458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e3c1f-8b0d-478a-b3fa-4a583cabbeaf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:20.997540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:21.007854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] MFA Rules not processed for user `8625d0b6889146dca447748ab1ae86b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:21.009425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.050671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.051324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.051658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.051933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.054911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.099367 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1059/2118] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:21.099946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.101437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.113813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:21.136294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.137961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.151871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1060/2119] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:21.152696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.154030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.167590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd74ad1-ab8f-4d59-91ce-9f8ff3bc4b1c service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.172499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.173121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.173402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.173678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.174372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.176305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.179435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55cc3d-74b7-4d3e-8a46-6663468f4811 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.184662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.185350 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.185744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.186525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] PATH_INFO: `/v3/users/ecc6773ab87249bbaffaaa49d8baf4d7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.187236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc6773ab87249bbaffaaa49d8baf4d7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.189305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.209177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1061/2120] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:21 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:21.212866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.242491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc6773ab87249bbaffaaa49d8baf4d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:21.252186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc6773ab87249bbaffaaa49d8baf4d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:21.268250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc6773ab87249bbaffaaa49d8baf4d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:21.269344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc6773ab87249bbaffaaa49d8baf4d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:21.280830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc6773ab87249bbaffaaa49d8baf4d7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:21.287997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1060/2121] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:21 2022] DELETE /identity/v3/users/ecc6773ab87249bbaffaaa49d8baf4d7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:21.302502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ae94e-16f0-42e5-a171-07b929f52a70 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.382701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.383418 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.384134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.384419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.385070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.387354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.416186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1062/2122] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:21 2022] POST /identity/v3/users => generated 308 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:21.420674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.422732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.437977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a73d8-e1e4-41c7-a822-078e56c6f307 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.443073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.443788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.443969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.444253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.444876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.446661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.461166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1061/2123] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:21.465302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.476700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fc2d9-c34b-4ee5-a654-1b0dbc922de3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.483336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.484273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.484701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.485074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] PATH_INFO: `/v3/projects/2ce185bc24f44018bd421c27b1d68b00/users/e2e01fea76dc4e04b9bf3b9b887be3d5/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.502228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce185bc24f44018bd421c27b1d68b00, user_id=e2e01fea76dc4e04b9bf3b9b887be3d5, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.505050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.523944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1063/2124] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:21 2022] PUT /identity/v3/projects/2ce185bc24f44018bd421c27b1d68b00/users/e2e01fea76dc4e04b9bf3b9b887be3d5/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:21.528811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.542250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de40294-43bd-4eed-96b7-1ddbb9348ba5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.546861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:21.643500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3afa9-ea76-49a9-b603-a211549ea7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.644717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3afa9-ea76-49a9-b603-a211549ea7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.645246 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3afa9-ea76-49a9-b603-a211549ea7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.645843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3afa9-ea76-49a9-b603-a211549ea7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.646624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3afa9-ea76-49a9-b603-a211549ea7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.648315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3afa9-ea76-49a9-b603-a211549ea7bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.661712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1062/2125] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:21.665839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.679999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210844-7eda-4b68-a6bf-2150ad5152a7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.686178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.687975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.688254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.688563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] PATH_INFO: `/v3/projects/2ce185bc24f44018bd421c27b1d68b00/users/e2e01fea76dc4e04b9bf3b9b887be3d5/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.690785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce185bc24f44018bd421c27b1d68b00, user_id=e2e01fea76dc4e04b9bf3b9b887be3d5, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.694127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.709847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1064/2126] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:21 2022] PUT /identity/v3/projects/2ce185bc24f44018bd421c27b1d68b00/users/e2e01fea76dc4e04b9bf3b9b887be3d5/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:21.720817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.721345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.721916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.746358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.748404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.772871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] MFA Rules not processed for user `e2e01fea76dc4e04b9bf3b9b887be3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:21.774142 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:21.778476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:21.876868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62295e4-6e19-4c66-829d-acaded937a92 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:21.886545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:21.887873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:21.888366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:21.888805 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] PATH_INFO: `/v3/projects/ca83afe67f424cf6a794e592eff9ca7c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:21.890287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca83afe67f424cf6a794e592eff9ca7c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:21.893066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:21.913387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1063/2127] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:21.924259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:21.925804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.003996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca83afe67f424cf6a794e592eff9ca7c', 'request_id': 'req-ee933df9-987f-41b1-ba13-37a9288f0190'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.012850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca83afe67f424cf6a794e592eff9ca7c', 'request_id': 'req-ee933df9-987f-41b1-ba13-37a9288f0190'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.023931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca83afe67f424cf6a794e592eff9ca7c', 'request_id': 'req-ee933df9-987f-41b1-ba13-37a9288f0190'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.044340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] The token cache is being invalidate because project ca83afe67f424cf6a794e592eff9ca7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:22.044685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.045651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ee933df9-987f-41b1-ba13-37a9288f0190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.052382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1065/2128] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:21 2022] DELETE /identity/v3/projects/ca83afe67f424cf6a794e592eff9ca7c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:22.053660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876d8a4-c4e4-47ae-8837-b9bd2bc0b65e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.054172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7f915-4500-4c71-8bf5-9b278187df23 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.054646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7f915-4500-4c71-8bf5-9b278187df23 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.055077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7f915-4500-4c71-8bf5-9b278187df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.095385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a9f7f915-4500-4c71-8bf5-9b278187df23 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:22.097285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a9f7f915-4500-4c71-8bf5-9b278187df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.115311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:22.125813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.126542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.127773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.128041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.130474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.194843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1066/2129] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:22.202524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.202710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.203050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.213685 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.215137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.226384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1064/2130] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:21 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:01:22.235229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:22.237996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.254156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1067/2131] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:22.256221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.268431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fd89b-e923-4d97-b425-512f7e5b7dd7 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.276242 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.277030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.277321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.277596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.278293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:22.278603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.280224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.295016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1065/2132] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:22.299517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.310271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc1612-1049-4656-9705-82a5c74823a4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.315816 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.316434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.317296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.317296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.317592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.319592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.364490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1068/2133] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:22.364621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.371010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341bf27-a06c-40eb-a341-3ae7bf5cc736 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.419967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.431638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.464967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e01b9-e81f-4cca-a780-f6962b0bbba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.465999 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e01b9-e81f-4cca-a780-f6962b0bbba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.466299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e01b9-e81f-4cca-a780-f6962b0bbba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.466579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e01b9-e81f-4cca-a780-f6962b0bbba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.467219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e01b9-e81f-4cca-a780-f6962b0bbba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.468067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:22.470290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e01b9-e81f-4cca-a780-f6962b0bbba5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.494248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1066/2134] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:22.495403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.495694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.495970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.523912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc4c9f-89a5-4933-89c1-790754bc5276 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.530057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] MFA Rules not processed for user `482233dae68d487ca9b2ff6c69684663`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:22.531852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.532659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.533631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.534241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.534710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] PATH_INFO: `/v3/users/8625d0b6889146dca447748ab1ae86b8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.536216 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8625d0b6889146dca447748ab1ae86b8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.538936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.594027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8625d0b6889146dca447748ab1ae86b8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.605950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:22.607333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8625d0b6889146dca447748ab1ae86b8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.610937 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1067/2135] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:22.611829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.632900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1f719-41ca-4a53-80a2-6a6821566a37 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.635323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8625d0b6889146dca447748ab1ae86b8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.636039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8625d0b6889146dca447748ab1ae86b8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.638399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93144a00-a94c-4dc9-9e33-faa1c8699411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.639179 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93144a00-a94c-4dc9-9e33-faa1c8699411 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.639519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93144a00-a94c-4dc9-9e33-faa1c8699411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.639798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93144a00-a94c-4dc9-9e33-faa1c8699411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.641137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93144a00-a94c-4dc9-9e33-faa1c8699411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.643173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93144a00-a94c-4dc9-9e33-faa1c8699411 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.644315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8625d0b6889146dca447748ab1ae86b8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:22.654567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1069/2136] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:22 2022] DELETE /identity/v3/users/8625d0b6889146dca447748ab1ae86b8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:22.655389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.656535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1068/2137] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:22.659556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.659848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.660075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.660172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.680201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:22.681398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.719633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde8941-4f04-45f3-b174-c0d02b1886af admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.723235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.723839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.724172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.724376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.726345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.744949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1069/2138] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:22.746144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.747427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.762846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644468c-4459-4cd6-a2f3-501ac2bd7db9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.768220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.768821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.769575 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.769864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.770541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.772130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.782718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1070/2139] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:22.783518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.785108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.786423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:22.790010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.791901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.802472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1070/2140] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:22.803248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.814507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31500f4-4707-4485-ba97-336ac13aa369 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.820798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f831a-bc89-4169-9f6e-af90d7bc1d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.821487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f831a-bc89-4169-9f6e-af90d7bc1d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.821762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f831a-bc89-4169-9f6e-af90d7bc1d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.822066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f831a-bc89-4169-9f6e-af90d7bc1d0f admin admin] PATH_INFO: `/v3/projects/ae2c357a67494ac3bd5de114971aea4c/users/fa82efe4966d4fb08d1536880b8cdc4c/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.837966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f831a-bc89-4169-9f6e-af90d7bc1d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2c357a67494ac3bd5de114971aea4c, user_id=fa82efe4966d4fb08d1536880b8cdc4c, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.841127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f831a-bc89-4169-9f6e-af90d7bc1d0f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:22.850935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92e28-d987-4419-8671-36753c7570b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:22.855002 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1071/2141] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:22 2022] PUT /identity/v3/projects/ae2c357a67494ac3bd5de114971aea4c/users/fa82efe4966d4fb08d1536880b8cdc4c/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:22.866252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.866540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.866814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.895746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] MFA Rules not processed for user `fa82efe4966d4fb08d1536880b8cdc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:22.897141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:22.914734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:22.915450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:22.915732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:22.916010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:22.977480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1072/2142] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:22.979910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:22.982406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:22.997365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:22.998769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.010365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1071/2143] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:23.055244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82219d0e-8d83-4e89-9511-d3b37c2a3c09 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.060175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.060784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.061057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.061331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.062607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.064506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.109777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1073/2144] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:22 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:23.110108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.118821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb100bc-f175-4ba1-8a47-653c6ffe14de service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.199542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.200238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.200583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.200924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.201676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.205922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.234641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1072/2145] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:23 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:23.236547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.248781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c873ae9-50b7-4f28-bd14-f5f92574d4d7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.254218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.254905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.255394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.255736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.256479 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.258625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.269220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1074/2146] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:23.273276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.284194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8460ed-69d0-4d34-9935-b39e856fcdd0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.288760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca1ced-f9f7-46f2-ba53-6b42eac2d5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.289357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca1ced-f9f7-46f2-ba53-6b42eac2d5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.289637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca1ced-f9f7-46f2-ba53-6b42eac2d5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.289930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca1ced-f9f7-46f2-ba53-6b42eac2d5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.290585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca1ced-f9f7-46f2-ba53-6b42eac2d5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.292158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca1ced-f9f7-46f2-ba53-6b42eac2d5d1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.305684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1073/2147] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:23.307165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.317747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505ff-7164-4edc-8879-ad1893b8f1c5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.322917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.323687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.323969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.324239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] PATH_INFO: `/v3/projects/c49d0705bcaf433fabe25d7e347e2a4c/users/50884f6b06e74a5d9c31f79bfbab27c7/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.340119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49d0705bcaf433fabe25d7e347e2a4c, user_id=50884f6b06e74a5d9c31f79bfbab27c7, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.343024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.356880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1075/2148] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:23 2022] PUT /identity/v3/projects/c49d0705bcaf433fabe25d7e347e2a4c/users/50884f6b06e74a5d9c31f79bfbab27c7/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:23.367622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.367901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.368180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.399930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] MFA Rules not processed for user `50884f6b06e74a5d9c31f79bfbab27c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:23.401302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:23.422212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.424219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:23.446132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:23.494231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1074/2149] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:23.495137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.537459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7151bf-df14-4624-927c-6ee47339f41c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.542567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.543337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.543550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.543910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] PATH_INFO: `/v3/projects/108f2369986c4d6eb65e285fa4e7e823` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.545364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108f2369986c4d6eb65e285fa4e7e823)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.548045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.575165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:23.595016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d74446-bcaf-48aa-82eb-30d6cc0a6fbe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.610903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108f2369986c4d6eb65e285fa4e7e823', 'request_id': 'req-dfd66dda-fbcb-469d-ade2-72396efea9b7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.618286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108f2369986c4d6eb65e285fa4e7e823', 'request_id': 'req-dfd66dda-fbcb-469d-ade2-72396efea9b7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.650028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108f2369986c4d6eb65e285fa4e7e823', 'request_id': 'req-dfd66dda-fbcb-469d-ade2-72396efea9b7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.665868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.667051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.667513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.667962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.668311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] The token cache is being invalidate because project 108f2369986c4d6eb65e285fa4e7e823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:23.668933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.669435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.669976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.672672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.673322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1076/2150] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:23 2022] DELETE /identity/v3/projects/108f2369986c4d6eb65e285fa4e7e823 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:23.676970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.678725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:23.696702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd66dda-fbcb-469d-ade2-72396efea9b7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.702453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.703428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.704124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.704382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] PATH_INFO: `/v3/users/e2e01fea76dc4e04b9bf3b9b887be3d5` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.705617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e01fea76dc4e04b9bf3b9b887be3d5)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.708800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.723574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1075/2151] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:23 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:23.729125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.730865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:23.761405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e01fea76dc4e04b9bf3b9b887be3d5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.771235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e01fea76dc4e04b9bf3b9b887be3d5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.785861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e01fea76dc4e04b9bf3b9b887be3d5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.786503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e01fea76dc4e04b9bf3b9b887be3d5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.794335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e01fea76dc4e04b9bf3b9b887be3d5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:23.802275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1077/2152] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:23 2022] DELETE /identity/v3/users/e2e01fea76dc4e04b9bf3b9b887be3d5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:23.802593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b382a60-4075-4024-aed6-7032203e018a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.873639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.874353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.874667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.875021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.875866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.878052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.916789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1076/2153] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:23 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:23.917152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.917152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:23.937570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7208f0-e64c-4872-b360-6c4abbafb3bf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:23.942809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:23.943336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:23.943621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:23.943897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:23.944547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:23.946237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:23.952163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:23.960992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1078/2154] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:23.965437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:23.993895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce166cdb-26fa-4e16-827a-a9eeeedf5dcb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.008530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.009376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.009843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.010156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] PATH_INFO: `/v3/projects/1f03e144ed1743b89fce9870e1705961/users/2c6d70cf35ac4a238122cd0c4834412c/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.029878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f03e144ed1743b89fce9870e1705961, user_id=2c6d70cf35ac4a238122cd0c4834412c, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.033271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.042083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:24.049435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1077/2155] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:23 2022] PUT /identity/v3/projects/1f03e144ed1743b89fce9870e1705961/users/2c6d70cf35ac4a238122cd0c4834412c/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:24.053649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.064960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eb4a1-e635-478f-8c7a-acfc9cf26a78 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.126012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.127989 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.128683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.129360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.129703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.130072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.130883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.132595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.140926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60cff5-c790-473f-8538-f3df829cccc1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.147055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.151225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1079/2156] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:24.151538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.151538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.151538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] PATH_INFO: `/v3/projects/2ce185bc24f44018bd421c27b1d68b00` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.151538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce185bc24f44018bd421c27b1d68b00)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.153514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.155116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.164913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c90aa9-3966-4938-91f5-26773f4ca02b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.169446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39def1-33d1-496f-9413-62aa79aa8773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.170120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39def1-33d1-496f-9413-62aa79aa8773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.170453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39def1-33d1-496f-9413-62aa79aa8773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.170729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39def1-33d1-496f-9413-62aa79aa8773 admin admin] PATH_INFO: `/v3/projects/1f03e144ed1743b89fce9870e1705961/users/2c6d70cf35ac4a238122cd0c4834412c/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.174433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39def1-33d1-496f-9413-62aa79aa8773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f03e144ed1743b89fce9870e1705961, user_id=2c6d70cf35ac4a238122cd0c4834412c, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.177066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39def1-33d1-496f-9413-62aa79aa8773 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.191270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1080/2157] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:24 2022] PUT /identity/v3/projects/1f03e144ed1743b89fce9870e1705961/users/2c6d70cf35ac4a238122cd0c4834412c/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:24.201166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb2bb-2efb-4282-b06f-9959f290c054 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.201445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb2bb-2efb-4282-b06f-9959f290c054 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.201731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb2bb-2efb-4282-b06f-9959f290c054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.206019 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce185bc24f44018bd421c27b1d68b00', 'request_id': 'req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:24.213346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce185bc24f44018bd421c27b1d68b00', 'request_id': 'req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:24.223827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce185bc24f44018bd421c27b1d68b00', 'request_id': 'req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:24.230620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-853eb2bb-2efb-4282-b06f-9959f290c054 None None] MFA Rules not processed for user `2c6d70cf35ac4a238122cd0c4834412c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:24.231923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-853eb2bb-2efb-4282-b06f-9959f290c054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.240358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] The token cache is being invalidate because project 2ce185bc24f44018bd421c27b1d68b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:24.240656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:24.241382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37b557d7-d3f2-4d45-aeee-38a7348b9b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:24.243052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1078/2158] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:24 2022] DELETE /identity/v3/projects/2ce185bc24f44018bd421c27b1d68b00 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:24.244375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.244729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.245014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.264411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] MFA Rules not processed for user `482233dae68d487ca9b2ff6c69684663`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:24.265684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.305802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1081/2159] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:24.306809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d2402-9634-49f3-8319-dbbeffc02b25 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.307098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d2402-9634-49f3-8319-dbbeffc02b25 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.307506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d2402-9634-49f3-8319-dbbeffc02b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.324087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1079/2160] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:24.325035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.326562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.327134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-663d2402-9634-49f3-8319-dbbeffc02b25 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:24.328668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-663d2402-9634-49f3-8319-dbbeffc02b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.391425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a54584-e32c-48e4-bc28-0828a10eec5c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.395182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.395782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.396063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.396362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.402090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.409110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1082/2161] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:24.424971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.425795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.425795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.450636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:24.452260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:24.453755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:24.462584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1083/2162] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:24.466547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.471878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:24.480383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.482194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cc5fd-44d4-4583-af3d-1695f32799f3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.482347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.489677 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.490477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.490821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.491178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.491887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:24.492255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.494051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.494593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1080/2163] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:24.512400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1084/2164] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:24.513507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.525484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d52a54-a939-4487-b60f-783b757f868a service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.530392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.531605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.531951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.532668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.532931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.535055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.564831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1081/2165] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:24.569126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.583335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb147-bf81-4429-9e59-4a8451963aac admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.659053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.659814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.660100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.660854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.661662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.664383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.694147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1085/2166] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:24.698526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.737305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4b3d2-70b7-4261-b091-c0297d2006e6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.756293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.756690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.756980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.757262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.757961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.759606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.783129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1082/2167] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:24.787658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.799305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8658f2b-4919-439c-b92a-efd9c7e6aa1c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.804659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b8cb4-21c7-4d28-bb4d-e51299f6f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.805384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8cb4-21c7-4d28-bb4d-e51299f6f10f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.805976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8cb4-21c7-4d28-bb4d-e51299f6f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.806165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8cb4-21c7-4d28-bb4d-e51299f6f10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.806913 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b8cb4-21c7-4d28-bb4d-e51299f6f10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.810095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b8cb4-21c7-4d28-bb4d-e51299f6f10f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.824530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:24.825032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1086/2168] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:24.857136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71100fd0-d9fd-40c3-a2ea-8cd1eba7c1c3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:24.872840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:24.874621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.874621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.874621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] PATH_INFO: `/v3/projects/26b5ba69f099446295672ecdea1338e7/users/d686e39fee004f89825be2436d35e13c/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.908540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b5ba69f099446295672ecdea1338e7, user_id=d686e39fee004f89825be2436d35e13c, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:24.917972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:24.937826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1083/2169] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:24 2022] PUT /identity/v3/projects/26b5ba69f099446295672ecdea1338e7/users/d686e39fee004f89825be2436d35e13c/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:24.948728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:24.949028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:24.949307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:24.986591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] MFA Rules not processed for user `d686e39fee004f89825be2436d35e13c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:24.988123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:25.132787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1087/2170] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:25.137037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.231494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eac207-607a-413c-94de-6b3b1e96d112 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.239998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.240670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.240949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.241348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.242083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.244052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:25.284163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1084/2171] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:25 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:25.288004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.316325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a440d7-2aa8-441a-8ebc-04012c32cd06 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.415346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:25.425457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.426382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.426880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.427240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.428435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.431140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:25.462104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1088/2172] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:25 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:25.466191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.478062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d889dd-9c21-41d2-8000-d2f06cafee26 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.485023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.486456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.487283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.487733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.489382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.491078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:25.502883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1085/2173] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:25.507240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.520178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9966-bdb0-43b1-850d-e9b75fcd878c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.526246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.527238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.527640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.528055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] PATH_INFO: `/v3/projects/165507d751c44715bee4bc25076b84e1/users/38baf9e0251f4dd6932cc98c735ba3a5/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.546860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165507d751c44715bee4bc25076b84e1, user_id=38baf9e0251f4dd6932cc98c735ba3a5, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.549667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:25.565484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1089/2174] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:25 2022] PUT /identity/v3/projects/165507d751c44715bee4bc25076b84e1/users/38baf9e0251f4dd6932cc98c735ba3a5/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:25.569584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.593500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e663f-94d5-463e-81fc-8cba9569c3dd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.650052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:25.657128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d1d98-892d-49b7-bad0-36b81096f187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.658714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d1d98-892d-49b7-bad0-36b81096f187 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.660105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d1d98-892d-49b7-bad0-36b81096f187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.661534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d1d98-892d-49b7-bad0-36b81096f187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.661534 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d1d98-892d-49b7-bad0-36b81096f187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.663478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d1d98-892d-49b7-bad0-36b81096f187 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:25.675859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1086/2175] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:25.681448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.694212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33460039-42ab-409d-942b-ea2b13451ac4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.701671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.702710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.703048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.703379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] PATH_INFO: `/v3/projects/165507d751c44715bee4bc25076b84e1/users/38baf9e0251f4dd6932cc98c735ba3a5/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.706283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165507d751c44715bee4bc25076b84e1, user_id=38baf9e0251f4dd6932cc98c735ba3a5, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.709998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:25.732412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1090/2176] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:25 2022] PUT /identity/v3/projects/165507d751c44715bee4bc25076b84e1/users/38baf9e0251f4dd6932cc98c735ba3a5/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:25.735632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.736731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.736731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.766319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] MFA Rules not processed for user `38baf9e0251f4dd6932cc98c735ba3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:25.767543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:25.850802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1087/2177] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:25.858088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:25.860657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:25.953967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851473b-a100-4aae-97de-298c6c0c6be2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:25.965073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:25.966148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:25.967674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:25.970173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:25.990974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:25.992502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:26.005544 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1091/2178] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:26.239679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:26.249738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4028f0-6f0b-4b4b-a1dd-f91e49e26117 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:26.253949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:26.254571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:26.254859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:26.255133 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:26.257570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:26.325809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:26.327277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff60b-e223-4ffe-93e2-554a563df286 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:26.336981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1088/2179] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:26.498948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:26.574614 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563d316-5c16-4530-bacc-adfc3f5c216e service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:26.580304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:26.581044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:26.581371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:26.581652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] PATH_INFO: `/v3/users/d686e39fee004f89825be2436d35e13c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:26.582746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d686e39fee004f89825be2436d35e13c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:26.584931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:26.640665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd686e39fee004f89825be2436d35e13c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:26.648834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd686e39fee004f89825be2436d35e13c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:26.650378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:26.664409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd686e39fee004f89825be2436d35e13c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:26.665138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd686e39fee004f89825be2436d35e13c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:26.672549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd686e39fee004f89825be2436d35e13c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:26.679435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1092/2180] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:26 2022] DELETE /identity/v3/users/d686e39fee004f89825be2436d35e13c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:26.684091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:26.684323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:26.684602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:26.715624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:26.717241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:26.796475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1089/2181] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:26.806386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:26.808068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:26.885883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a011d0-fa95-4575-b5ba-ff5d97659bd7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:26.893126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:26.894086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:26.894555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:26.894999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:26.910293 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:26.912036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:26.937543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1093/2182] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:26.993893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:26.995466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:27.018405 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:27.084706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a6b3c-ced3-4b6d-9f46-d6dde86b1cc3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:27.090628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:27.091255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:27.091533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:27.091935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:27.094290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:27.173108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:27.174639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:27.184573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1090/2183] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:27.435111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:27.436576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:27.477063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740d75b-881e-403e-89a4-d6b0c0cd59b9 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:27.482921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:27.483644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:27.483964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:27.484269 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] PATH_INFO: `/v3/projects/26b5ba69f099446295672ecdea1338e7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:27.485516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b5ba69f099446295672ecdea1338e7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:27.487552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:27.551237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b5ba69f099446295672ecdea1338e7', 'request_id': 'req-4f8b8434-bb0d-431b-a686-b9ea3a531a37'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.560491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b5ba69f099446295672ecdea1338e7', 'request_id': 'req-4f8b8434-bb0d-431b-a686-b9ea3a531a37'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.568663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b5ba69f099446295672ecdea1338e7', 'request_id': 'req-4f8b8434-bb0d-431b-a686-b9ea3a531a37'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.584586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] The token cache is being invalidate because project 26b5ba69f099446295672ecdea1338e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:27.584880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.585535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.587077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1094/2184] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:27 2022] DELETE /identity/v3/projects/26b5ba69f099446295672ecdea1338e7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:27.590967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:27.614342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157a2ac-4509-46da-8bb2-393541d21fd1 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:27.701452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:27.702089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:27.702374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:27.703180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] PATH_INFO: `/v3/users/38baf9e0251f4dd6932cc98c735ba3a5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:27.704391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38baf9e0251f4dd6932cc98c735ba3a5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:27.706575 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:27.769612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38baf9e0251f4dd6932cc98c735ba3a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.779369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38baf9e0251f4dd6932cc98c735ba3a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.792819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:27.794782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38baf9e0251f4dd6932cc98c735ba3a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.795518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38baf9e0251f4dd6932cc98c735ba3a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.804948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-54b471f3-a31d-480e-afa6-54673dd45ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38baf9e0251f4dd6932cc98c735ba3a5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:27.811014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1091/2185] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:27 2022] DELETE /identity/v3/users/38baf9e0251f4dd6932cc98c735ba3a5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:28.115689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.117530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:28.130983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b8434-bb0d-431b-a686-b9ea3a531a37 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.185521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:28.192866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.193572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.193874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.194412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] PATH_INFO: `/v3/projects/165507d751c44715bee4bc25076b84e1` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.195981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165507d751c44715bee4bc25076b84e1)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.197926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.245963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165507d751c44715bee4bc25076b84e1', 'request_id': 'req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:28.252495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165507d751c44715bee4bc25076b84e1', 'request_id': 'req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:28.259235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165507d751c44715bee4bc25076b84e1', 'request_id': 'req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:28.276048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] The token cache is being invalidate because project 165507d751c44715bee4bc25076b84e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:28.276275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:28.277108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ab797cd-ed0f-48af-b7c1-3b470b3e5d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:28.279201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1095/2186] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:28 2022] DELETE /identity/v3/projects/165507d751c44715bee4bc25076b84e1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:28.286595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.286595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.286884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.312001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:28.313660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:28.384267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1092/2187] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:28.392376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.392657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.392929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.413871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:28.415202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:28.427071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1096/2188] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:28.428309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.440206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fbd33-b982-4aeb-bd7e-6d7c0d50334a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.446020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.446761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.447095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.447521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.448363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:28.448805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.451430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.467294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1093/2189] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:28.472015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.486085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8381d3-0d8c-489a-81fc-94d455b355a3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.492442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774e3fe-e574-4e7b-b739-f0b5f356a973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.493195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e3fe-e574-4e7b-b739-f0b5f356a973 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.493468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e3fe-e574-4e7b-b739-f0b5f356a973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.493743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e3fe-e574-4e7b-b739-f0b5f356a973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.494439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774e3fe-e574-4e7b-b739-f0b5f356a973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.496698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774e3fe-e574-4e7b-b739-f0b5f356a973 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.528880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1097/2190] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:28 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:28.533830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.546564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeaa1a-ceb9-4c93-af49-7e8f6a469392 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.651871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.652646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.653619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.654205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.657509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.662504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.697920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35334abd-dd43-4351-a2e8-ed20549ea7af None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.699218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35334abd-dd43-4351-a2e8-ed20549ea7af None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.699537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35334abd-dd43-4351-a2e8-ed20549ea7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.716630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1094/2191] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:28 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:28.731379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.738686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-35334abd-dd43-4351-a2e8-ed20549ea7af None None] MFA Rules not processed for user `67f6f1b1653246fb82e9d48813db8701`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:28.740054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-35334abd-dd43-4351-a2e8-ed20549ea7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:28.745528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77e556-5e7d-48b5-aa85-af4177014a3f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.754480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.755243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.755421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.755688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.756334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.758462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.779593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1095/2192] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:28.781529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.797411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b75e1-2f73-4096-8b46-b66a74a0ff58 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.804782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.805597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.806085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.806973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.808481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.810990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.830552 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1096/2193] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:28.830675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1098/2194] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:28.833107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:28.845014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db104b1-35dc-46d6-8c9f-b176aa1ee576 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:28.854076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:28.855711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.855988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.856261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] PATH_INFO: `/v3/projects/02ef90f8b3e044debb32e3c15d3723c7/users/2da5895e063849958ab9aec896db0f5e/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.875320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ef90f8b3e044debb32e3c15d3723c7, user_id=2da5895e063849958ab9aec896db0f5e, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:28.878181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:28.897645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1097/2195] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:28 2022] PUT /identity/v3/projects/02ef90f8b3e044debb32e3c15d3723c7/users/2da5895e063849958ab9aec896db0f5e/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:28.907830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:28.908190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:28.908534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:28.954077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] MFA Rules not processed for user `2da5895e063849958ab9aec896db0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:28.955503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:29.033128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:29.044670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1099/2196] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:29.049306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.071445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:29.121716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c817cec-ffe2-4850-bb98-16ad75bc5062 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.128052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.128741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.129125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.129472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.130219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.132238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.166589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1098/2197] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:29 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:29.167867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.183511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec266b-4b9b-4b35-b7cd-e98add663de2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.253446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.254241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.254601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.254947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.255661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.257717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.301063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1100/2198] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:29 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:29.302681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.328086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4366e4-61ca-49c7-b7c5-86bd86000040 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.333246 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.334000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.334284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.334555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.335761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.337185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.349512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1099/2199] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:29.353871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.366746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ed7f-c316-4685-a892-40a5fcddb4a3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.372420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.373649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.374070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.375155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] PATH_INFO: `/v3/projects/e4448a111dab422ea4ad58d3e2cb86a2/users/b65ebf6873d14967a33fb2ea82bb44c9/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.391093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4448a111dab422ea4ad58d3e2cb86a2, user_id=b65ebf6873d14967a33fb2ea82bb44c9, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.394047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.412382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1101/2200] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:29 2022] PUT /identity/v3/projects/e4448a111dab422ea4ad58d3e2cb86a2/users/b65ebf6873d14967a33fb2ea82bb44c9/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:29.416426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.428419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736063-9717-46d0-9623-d36e5254110a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.490727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5116f85-25fc-4e66-bea2-133f19185784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.491334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5116f85-25fc-4e66-bea2-133f19185784 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.491628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5116f85-25fc-4e66-bea2-133f19185784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.491909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5116f85-25fc-4e66-bea2-133f19185784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.492559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5116f85-25fc-4e66-bea2-133f19185784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.494188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5116f85-25fc-4e66-bea2-133f19185784 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.504954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1100/2201] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:29.509073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.519985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407cdfe-4b2c-449e-b63b-d8f587b08098 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.524860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.525591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.525964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.526311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] PATH_INFO: `/v3/projects/e4448a111dab422ea4ad58d3e2cb86a2/users/b65ebf6873d14967a33fb2ea82bb44c9/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.528297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4448a111dab422ea4ad58d3e2cb86a2, user_id=b65ebf6873d14967a33fb2ea82bb44c9, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.531786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.543492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1102/2202] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:29 2022] PUT /identity/v3/projects/e4448a111dab422ea4ad58d3e2cb86a2/users/b65ebf6873d14967a33fb2ea82bb44c9/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:29.555146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.555454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.555784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.583265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] MFA Rules not processed for user `b65ebf6873d14967a33fb2ea82bb44c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:29.584463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:29.664147 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1101/2203] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:29.677130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.679139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:29.760992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95c946-d54d-409c-b7b8-29093fb1e096 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.765128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f15aa-1b32-4447-9c82-42f9016d53bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.766126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f15aa-1b32-4447-9c82-42f9016d53bd service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.766419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f15aa-1b32-4447-9c82-42f9016d53bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.766702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f15aa-1b32-4447-9c82-42f9016d53bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.778664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f15aa-1b32-4447-9c82-42f9016d53bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.781757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f15aa-1b32-4447-9c82-42f9016d53bd service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:29.799282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1103/2204] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:29.882967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:29.900594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:29.966682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71972091-7003-44d9-b15e-6d71ca3cdc7a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:29.972231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:29.972921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:29.973267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:29.973609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] PATH_INFO: `/v3/users/2da5895e063849958ab9aec896db0f5e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:29.974758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da5895e063849958ab9aec896db0f5e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:29.977315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:30.026315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da5895e063849958ab9aec896db0f5e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.037308 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da5895e063849958ab9aec896db0f5e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.052163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da5895e063849958ab9aec896db0f5e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.052856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da5895e063849958ab9aec896db0f5e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.060434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da5895e063849958ab9aec896db0f5e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.067279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1102/2205] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:29 2022] DELETE /identity/v3/users/2da5895e063849958ab9aec896db0f5e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:30.072049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:30.072366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:30.072796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:30.094388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:30.095606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:30.142327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:30.161159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1104/2206] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:30.171418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:30.172803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:30.232855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9ec5-fa3b-4706-8a7d-9c971d63a620 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:30.239569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:30.239569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:30.239785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:30.239869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:30.251951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:30.253432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:30.262863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1103/2207] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:30.610178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:30.611600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:30.626647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f820db7-b4a6-4565-80ac-0992c2cc36bd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:30.631928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:30.632704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:30.633050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:30.633622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] PATH_INFO: `/v3/projects/02ef90f8b3e044debb32e3c15d3723c7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:30.635779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ef90f8b3e044debb32e3c15d3723c7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:30.638183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:30.707548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ef90f8b3e044debb32e3c15d3723c7', 'request_id': 'req-02cc09d7-18b7-46ee-8d99-a2c533705c9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.716455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ef90f8b3e044debb32e3c15d3723c7', 'request_id': 'req-02cc09d7-18b7-46ee-8d99-a2c533705c9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.724268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ef90f8b3e044debb32e3c15d3723c7', 'request_id': 'req-02cc09d7-18b7-46ee-8d99-a2c533705c9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.738154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] The token cache is being invalidate because project 02ef90f8b3e044debb32e3c15d3723c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:30.738971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.739598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.741534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1105/2208] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:30 2022] DELETE /identity/v3/projects/02ef90f8b3e044debb32e3c15d3723c7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:30.745503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:30.755917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725543f-dae1-4374-9aa6-ef02af43d705 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:30.794568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:30.825520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:30.826192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:30.826478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:30.826749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] PATH_INFO: `/v3/users/b65ebf6873d14967a33fb2ea82bb44c9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:30.828071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65ebf6873d14967a33fb2ea82bb44c9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:30.830132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:30.882370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65ebf6873d14967a33fb2ea82bb44c9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.892071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65ebf6873d14967a33fb2ea82bb44c9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.912234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65ebf6873d14967a33fb2ea82bb44c9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.913318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65ebf6873d14967a33fb2ea82bb44c9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.935494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65ebf6873d14967a33fb2ea82bb44c9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:30.944543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1104/2209] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:30 2022] DELETE /identity/v3/users/b65ebf6873d14967a33fb2ea82bb44c9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:31.242899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.244533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.260270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc09d7-18b7-46ee-8d99-a2c533705c9a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.314255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.315701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.335838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.336991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.337375 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.337752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] PATH_INFO: `/v3/projects/e4448a111dab422ea4ad58d3e2cb86a2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.339021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4448a111dab422ea4ad58d3e2cb86a2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.341040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.389465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad91c4-20c6-4222-af3b-e1fe9e48a305 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.394199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4448a111dab422ea4ad58d3e2cb86a2', 'request_id': 'req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.400092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4448a111dab422ea4ad58d3e2cb86a2', 'request_id': 'req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.409923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4448a111dab422ea4ad58d3e2cb86a2', 'request_id': 'req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.424441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] The token cache is being invalidate because project e4448a111dab422ea4ad58d3e2cb86a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:31.424766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.425436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c1f4aede-db67-4a6f-950a-a7e1ad77e63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.427184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1106/2210] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:31 2022] DELETE /identity/v3/projects/e4448a111dab422ea4ad58d3e2cb86a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:31.432292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887093cf-ffc4-4406-ad68-1bdf61b40d67 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.432580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887093cf-ffc4-4406-ad68-1bdf61b40d67 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.432859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887093cf-ffc4-4406-ad68-1bdf61b40d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.449974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:31.454621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.455397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.455742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.456082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] PATH_INFO: `/v3/users/fa82efe4966d4fb08d1536880b8cdc4c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.457577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa82efe4966d4fb08d1536880b8cdc4c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.459725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.463827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:31.469056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-887093cf-ffc4-4406-ad68-1bdf61b40d67 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:31.470646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-887093cf-ffc4-4406-ad68-1bdf61b40d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.480135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1107/2211] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:31.487494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.487779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.488059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.519359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:31.521718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.531782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa82efe4966d4fb08d1536880b8cdc4c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.539282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1108/2212] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:31.542612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa82efe4966d4fb08d1536880b8cdc4c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.545516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.556211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13779b-4c10-4d24-bf5c-d62df259fa89 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.556385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa82efe4966d4fb08d1536880b8cdc4c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.557004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa82efe4966d4fb08d1536880b8cdc4c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.563138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.563688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.563969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.564238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.564945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c399a734-ca38-410e-a56f-3e05e0ff6bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa82efe4966d4fb08d1536880b8cdc4c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:31.565072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:31.565345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.567402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.572494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1105/2213] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:31 2022] DELETE /identity/v3/users/fa82efe4966d4fb08d1536880b8cdc4c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:31.587250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1109/2214] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:31.589021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.589312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.589592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.591034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.593824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.610518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b328df-574a-4fe6-b64c-66bd08e6af4d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.630234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:31.631800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.735004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1106/2215] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:31.742273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.743123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.744260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.745013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.745845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.746976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.750141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:31.751818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.801879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1110/2216] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:31 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:31.806327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.843944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6265b1a-b8c1-4c66-9660-2de114ce978b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.898522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:31.902506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb369bc4-1c74-4cc5-9b7d-e8b2ed183a41 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.913044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.913813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.914327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.914680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.915586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.918040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.918174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.918550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.918899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.919248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.929486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.931118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.942927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1107/2217] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:01:31.948540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1111/2218] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:31 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:31.949796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:31.960623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec466d1-7e38-495e-b3af-35848ad358a8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:31.966352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:31.967089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:31.967483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:31.967866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:31.968641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:31.970395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:31.986082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1108/2219] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:31.987825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.000983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3ff9-2d8d-4cff-8d7c-b91005a9ad86 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.007452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846dfea-f708-4430-bcbc-88a4d0bff15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.008683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846dfea-f708-4430-bcbc-88a4d0bff15f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.009128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846dfea-f708-4430-bcbc-88a4d0bff15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.009434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846dfea-f708-4430-bcbc-88a4d0bff15f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.010201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846dfea-f708-4430-bcbc-88a4d0bff15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.012480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846dfea-f708-4430-bcbc-88a4d0bff15f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.036468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1112/2220] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:32.040041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.053862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb2623-769c-4338-8192-29d479bfa480 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.059275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.060232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.060684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.061116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] PATH_INFO: `/v3/projects/838b334789354457a5f24fb20ab2ab48/users/882e4d88a7c04828949e8ae4b4a509df/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.079061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=838b334789354457a5f24fb20ab2ab48, user_id=882e4d88a7c04828949e8ae4b4a509df, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.082446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.098737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1109/2221] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:32 2022] PUT /identity/v3/projects/838b334789354457a5f24fb20ab2ab48/users/882e4d88a7c04828949e8ae4b4a509df/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:32.108940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.109226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.109502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.157285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] MFA Rules not processed for user `882e4d88a7c04828949e8ae4b4a509df`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:32.158624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:32.193458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:32.246225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1113/2222] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:32.251450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.345163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.347058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:32.380502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de32b32-08e1-4809-8be5-b428dfd7f327 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.385509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.386330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.386716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.387095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.387848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.391225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.445623 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1110/2223] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:32 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:32.449790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.452354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e472-ca28-4c34-9eee-6cac66bb07eb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.535792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.536444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.536730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.537007 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] PATH_INFO: `/v3/projects/ae2c357a67494ac3bd5de114971aea4c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.537282 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb795e59-647a-4bc1-aeea-2f46b8dd0d4e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.538351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2c357a67494ac3bd5de114971aea4c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.540632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.542891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.543731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.544112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.544506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.545254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.547927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.580668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1111/2224] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:32 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:32.584450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.596080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882550c-f597-499b-aada-614bc67d2479 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.601834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2c357a67494ac3bd5de114971aea4c', 'request_id': 'req-3d60e82d-5b0e-4a49-9150-c0f623a24220'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.610277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2c357a67494ac3bd5de114971aea4c', 'request_id': 'req-3d60e82d-5b0e-4a49-9150-c0f623a24220'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.616469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:32.617517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2c357a67494ac3bd5de114971aea4c', 'request_id': 'req-3d60e82d-5b0e-4a49-9150-c0f623a24220'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.637973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] The token cache is being invalidate because project ae2c357a67494ac3bd5de114971aea4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:32.638270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.638915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.640478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1114/2225] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:32 2022] DELETE /identity/v3/projects/ae2c357a67494ac3bd5de114971aea4c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:32.644489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.645863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:32.657140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:32.663468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.664189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.664586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.664866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.665781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.667536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.680735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1112/2226] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:32.685044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.687574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:32.701158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfdb6f-b0e8-47bf-ab7a-7402cdb88c09 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.708408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.709158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.709564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.709940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.710816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.713034 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.722004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60e82d-5b0e-4a49-9150-c0f623a24220 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.725220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1113/2227] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:32.728533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.728533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.728533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.728989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] PATH_INFO: `/v3/users/50884f6b06e74a5d9c31f79bfbab27c7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.730029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=50884f6b06e74a5d9c31f79bfbab27c7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.730157 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.732003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.742283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bbd4-cd1a-475c-af69-5d375ee7ab21 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:32.748117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520fb96-d668-465f-b537-8610c2448f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:32.748889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520fb96-d668-465f-b537-8610c2448f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.749474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520fb96-d668-465f-b537-8610c2448f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.749783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520fb96-d668-465f-b537-8610c2448f4b admin admin] PATH_INFO: `/v3/projects/f19d52bf09ee45718b8368f0d5c163ad/users/60c1de480ddb48d69f217926ab1344a6/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.771295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520fb96-d668-465f-b537-8610c2448f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19d52bf09ee45718b8368f0d5c163ad, user_id=60c1de480ddb48d69f217926ab1344a6, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:32.774090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520fb96-d668-465f-b537-8610c2448f4b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:32.793035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50884f6b06e74a5d9c31f79bfbab27c7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.793404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1114/2228] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:32 2022] PUT /identity/v3/projects/f19d52bf09ee45718b8368f0d5c163ad/users/60c1de480ddb48d69f217926ab1344a6/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:32.805620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50884f6b06e74a5d9c31f79bfbab27c7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.807125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:32.807433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:32.807913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:32.823473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50884f6b06e74a5d9c31f79bfbab27c7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.824489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50884f6b06e74a5d9c31f79bfbab27c7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.833291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50884f6b06e74a5d9c31f79bfbab27c7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:32.842229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1115/2229] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:32 2022] DELETE /identity/v3/users/50884f6b06e74a5d9c31f79bfbab27c7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:32.848491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] MFA Rules not processed for user `60c1de480ddb48d69f217926ab1344a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:32.849747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:32.940401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1115/2230] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:32.945275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:32.947893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:33.027884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a7fc9-8689-46cc-b67b-ba187ac0aa2a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.033371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.034016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.034330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.034597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.035241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.037328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.070603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1116/2231] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:32 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:01:33.075795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.088573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45842cdd-c932-41c4-95a4-4c84c5ee9b4e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.162840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.163658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.164025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.164483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.178068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.178068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.187390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:33.196854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1116/2232] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:33 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:33.198188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.209143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6a7e-40f3-4c77-9263-5c4cd049e557 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.214624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.215434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.215718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.216004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.216765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.219087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.225097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:33.235091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1117/2233] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:33.236033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.247543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fddd9-35a9-4fad-9254-72ff8f3d5ca9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.261574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.262292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.262593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.262868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] PATH_INFO: `/v3/projects/c24a2159b30040b28596278233d87a9d/users/c422798ac6ca447195307b1e5db6bce0/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.278733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24a2159b30040b28596278233d87a9d, user_id=c422798ac6ca447195307b1e5db6bce0, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.281409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.295137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1117/2234] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:33 2022] PUT /identity/v3/projects/c24a2159b30040b28596278233d87a9d/users/c422798ac6ca447195307b1e5db6bce0/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:33.299070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.308963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4809d-b0b3-4bd7-ae43-be11f5d255d5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.365841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.366477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.366753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.367158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.367803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.369436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.384707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1118/2235] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:33.386006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.397803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbb928-6e1e-4a60-9820-b3ea4b5f07c4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.402814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e76241-d566-4535-bc88-eff420f56ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.403464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e76241-d566-4535-bc88-eff420f56ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.403740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e76241-d566-4535-bc88-eff420f56ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.404009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e76241-d566-4535-bc88-eff420f56ad0 admin admin] PATH_INFO: `/v3/projects/c24a2159b30040b28596278233d87a9d/users/c422798ac6ca447195307b1e5db6bce0/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.406122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e76241-d566-4535-bc88-eff420f56ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24a2159b30040b28596278233d87a9d, user_id=c422798ac6ca447195307b1e5db6bce0, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.408743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e76241-d566-4535-bc88-eff420f56ad0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.420403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1118/2236] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:33 2022] PUT /identity/v3/projects/c24a2159b30040b28596278233d87a9d/users/c422798ac6ca447195307b1e5db6bce0/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:33.429246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.431420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:33.435378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.435799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.436150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.445123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab5e7f-629c-4dfe-8d25-5e746c2a5713 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.492807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] MFA Rules not processed for user `c422798ac6ca447195307b1e5db6bce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:33.495324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:33.538603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.539339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.539678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.540070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] PATH_INFO: `/v3/projects/c49d0705bcaf433fabe25d7e347e2a4c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.541272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49d0705bcaf433fabe25d7e347e2a4c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.543718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.603322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1119/2237] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:33.611449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.615401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:33.625368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49d0705bcaf433fabe25d7e347e2a4c', 'request_id': 'req-914a544c-3276-4a4d-8ae6-50c5ad766b7d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.634078 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49d0705bcaf433fabe25d7e347e2a4c', 'request_id': 'req-914a544c-3276-4a4d-8ae6-50c5ad766b7d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.645055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49d0705bcaf433fabe25d7e347e2a4c', 'request_id': 'req-914a544c-3276-4a4d-8ae6-50c5ad766b7d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.665629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] The token cache is being invalidate because project c49d0705bcaf433fabe25d7e347e2a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:33.666117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.666899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.668452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1119/2238] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:33 2022] DELETE /identity/v3/projects/c49d0705bcaf433fabe25d7e347e2a4c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:33.683410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:33.685090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:33.701859 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6ebd5-7df8-4716-881c-abec41259485 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.744003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:33.774797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.775446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.775692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.775962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.779658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:33.787966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a544c-3276-4a4d-8ae6-50c5ad766b7d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:33.793457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:33.794134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:33.794420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:33.794697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] PATH_INFO: `/v3/users/2c6d70cf35ac4a238122cd0c4834412c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:33.795921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6d70cf35ac4a238122cd0c4834412c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.798814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.811195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:33.861220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6d70cf35ac4a238122cd0c4834412c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.870386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6d70cf35ac4a238122cd0c4834412c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.873884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:33.876415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:33.885866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6d70cf35ac4a238122cd0c4834412c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.886665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6d70cf35ac4a238122cd0c4834412c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.890001 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1120/2239] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:01:33.894845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6d70cf35ac4a238122cd0c4834412c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:33.901693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1120/2240] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:33 2022] DELETE /identity/v3/users/2c6d70cf35ac4a238122cd0c4834412c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:34.530300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:34.546101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22103-870e-4f5b-883c-7280c6e9c8d0 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:34.606457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:34.608386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:34.620573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:34.621548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:34.621971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:34.622355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] PATH_INFO: `/v3/projects/1f03e144ed1743b89fce9870e1705961` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:34.623607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f03e144ed1743b89fce9870e1705961)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:34.625649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:34.676925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bf1d5-2153-4e6b-8c17-15a6eb3894f7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:34.683355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f03e144ed1743b89fce9870e1705961', 'request_id': 'req-49e1ca6d-bf2f-4727-b165-094adb60ca10'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:34.690309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f03e144ed1743b89fce9870e1705961', 'request_id': 'req-49e1ca6d-bf2f-4727-b165-094adb60ca10'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:34.699292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f03e144ed1743b89fce9870e1705961', 'request_id': 'req-49e1ca6d-bf2f-4727-b165-094adb60ca10'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:34.721020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] The token cache is being invalidate because project 1f03e144ed1743b89fce9870e1705961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:34.721585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:34.722483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-49e1ca6d-bf2f-4727-b165-094adb60ca10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:34.728509 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1121/2241] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:34 2022] DELETE /identity/v3/projects/1f03e144ed1743b89fce9870e1705961 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:34.731029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:34.731502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:34.732050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:34.755373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:34.756129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:34.756470 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:34.756746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:34.757132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:34.758579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:34.759276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:34.775996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:34.843899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:34.864101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:34.866453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53629556-b2d6-4f93-a55b-dcfec52d8405 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:34.879618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1121/2242] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:01:34.883166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1122/2243] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:34.892118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:34.892468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:34.892684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:34.922887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:34.924273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:34.939161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1122/2244] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:34.943373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:34.954593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914da934-7d2c-48f2-8cd2-2dd02cf30e5d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:34.959436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:34.960100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:34.960386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:34.960663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:34.961371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:34.961695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:34.963531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:34.975857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1123/2245] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:34.980071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:34.989525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2967a-c55d-497c-8811-25aaf0ca434f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:34.994103 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:34.994700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:34.994978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:34.995250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:34.995884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:34.997777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:35.027053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1123/2246] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:34 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:35.031239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:35.042601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc6bff-0d5d-4695-98f8-24a84767c1ae admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:35.103065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:35.103665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:35.103945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:35.104219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:35.104851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:35.107049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:35.131353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1124/2247] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:35 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:35.137885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:35.164238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f460f-cc27-49df-95c0-f9d16b14f477 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:35.168877 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:35.169481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:35.169761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:35.170071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:35.170718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:35.172852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:35.183399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1124/2248] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:35.187837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:35.200287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c321846-cd0f-4798-9bb4-f583be9f2c72 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:35.218646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ead4d-d0f7-4de0-ac19-e2eb1df2c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:35.219260 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ead4d-d0f7-4de0-ac19-e2eb1df2c7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:35.219543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ead4d-d0f7-4de0-ac19-e2eb1df2c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:35.219818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ead4d-d0f7-4de0-ac19-e2eb1df2c7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:35.220469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ead4d-d0f7-4de0-ac19-e2eb1df2c7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:35.222091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ead4d-d0f7-4de0-ac19-e2eb1df2c7e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:35.233530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1125/2249] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:35.238676 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:35.248982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d13bf-cd5b-4f23-9be4-88c1d105c154 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:35.254154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:35.254795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:35.255080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:35.255352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] PATH_INFO: `/v3/projects/cb11a698923049a9bc314419acee7466/users/3d79d6bc06124a6f93904fb6e5743ade/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:35.270313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb11a698923049a9bc314419acee7466, user_id=3d79d6bc06124a6f93904fb6e5743ade, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:35.272970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:35.286487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1125/2250] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:35 2022] PUT /identity/v3/projects/cb11a698923049a9bc314419acee7466/users/3d79d6bc06124a6f93904fb6e5743ade/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:35.297169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:35.297457 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:35.297733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:35.328673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] MFA Rules not processed for user `3d79d6bc06124a6f93904fb6e5743ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:35.329914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:35.403697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1126/2251] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:35.415213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:35.417020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:35.475009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d1aa-ec4c-43e1-8094-4f45577d01ad admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:35.479287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fac28-02b4-4848-831e-98464818843b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:35.479894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fac28-02b4-4848-831e-98464818843b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:35.480174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fac28-02b4-4848-831e-98464818843b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:35.480449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fac28-02b4-4848-831e-98464818843b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:35.489591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fac28-02b4-4848-831e-98464818843b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:35.491072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fac28-02b4-4848-831e-98464818843b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:35.500640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1126/2252] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:39.115485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:39.125854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b9021-03fc-4e1b-a32a-6a1d2437c1c9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:39.129420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:39.130052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:39.130332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:39.130608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:39.132335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:39.174561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:39.204365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:39.207909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbde025-8107-4f41-bc5e-f56f9bf72174 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:39.221087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1127/2253] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:41.506417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fac28-02b4-4848-831e-98464818843b service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:41.508228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-681fac28-02b4-4848-831e-98464818843b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:41.573458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:41.582123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fac28-02b4-4848-831e-98464818843b service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:41.586981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:41.587644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:41.587927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:41.588203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] PATH_INFO: `/v3/users/882e4d88a7c04828949e8ae4b4a509df` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:41.589497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882e4d88a7c04828949e8ae4b4a509df)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:41.592645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:41.662322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882e4d88a7c04828949e8ae4b4a509df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:41.669640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882e4d88a7c04828949e8ae4b4a509df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:41.682653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882e4d88a7c04828949e8ae4b4a509df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:41.683331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882e4d88a7c04828949e8ae4b4a509df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:41.691691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882e4d88a7c04828949e8ae4b4a509df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:41.698447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1127/2254] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:41 2022] DELETE /identity/v3/users/882e4d88a7c04828949e8ae4b4a509df => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:41.703814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:41.704484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:41.707009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:41.733787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:41.735024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:41.799988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1128/2255] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:41.809675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:41.811178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:41.870804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19548a18-28ac-4861-8bd3-92af42658593 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:41.874892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:41.875545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:41.875827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:41.876096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:41.885882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:41.887641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:41.899117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1128/2256] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:42.187507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:42.189505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:42.202267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6990e-fbc9-4f9b-98b2-66c583e32fe0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:42.207015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:42.207703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:42.207922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:42.208223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] PATH_INFO: `/v3/projects/838b334789354457a5f24fb20ab2ab48` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:42.209448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=838b334789354457a5f24fb20ab2ab48)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:42.211489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:42.282051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838b334789354457a5f24fb20ab2ab48', 'request_id': 'req-2c8159a2-0284-4707-9952-fe72fd49bb0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.288695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838b334789354457a5f24fb20ab2ab48', 'request_id': 'req-2c8159a2-0284-4707-9952-fe72fd49bb0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.297098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838b334789354457a5f24fb20ab2ab48', 'request_id': 'req-2c8159a2-0284-4707-9952-fe72fd49bb0e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.314437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] The token cache is being invalidate because project 838b334789354457a5f24fb20ab2ab48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:42.314732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.315424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.316920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1129/2257] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:42 2022] DELETE /identity/v3/projects/838b334789354457a5f24fb20ab2ab48 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:42.321657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:42.336871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035028ab-9e7d-4fb6-a02a-3e4a9e30fd8e service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:42.397890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:42.398480 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:42.398762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:42.399035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] PATH_INFO: `/v3/users/60c1de480ddb48d69f217926ab1344a6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:42.400160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60c1de480ddb48d69f217926ab1344a6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:42.402365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:42.429436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:42.458528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60c1de480ddb48d69f217926ab1344a6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.469074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60c1de480ddb48d69f217926ab1344a6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.492557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60c1de480ddb48d69f217926ab1344a6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.493315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60c1de480ddb48d69f217926ab1344a6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.501012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60c1de480ddb48d69f217926ab1344a6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:42.513491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1129/2258] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:42 2022] DELETE /identity/v3/users/60c1de480ddb48d69f217926ab1344a6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:42.886962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:42.889586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:42.908874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8159a2-0284-4707-9952-fe72fd49bb0e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:42.982154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:42.982792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:42.983071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:42.983341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] PATH_INFO: `/v3/projects/f19d52bf09ee45718b8368f0d5c163ad` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:42.984477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19d52bf09ee45718b8368f0d5c163ad)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:42.986443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:42.989274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:43.039630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19d52bf09ee45718b8368f0d5c163ad', 'request_id': 'req-9d8735a4-2caf-432f-a118-ebaccae9008f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.045415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19d52bf09ee45718b8368f0d5c163ad', 'request_id': 'req-9d8735a4-2caf-432f-a118-ebaccae9008f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.059687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19d52bf09ee45718b8368f0d5c163ad', 'request_id': 'req-9d8735a4-2caf-432f-a118-ebaccae9008f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.073257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] The token cache is being invalidate because project f19d52bf09ee45718b8368f0d5c163ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:43.073599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.074311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.075984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1130/2259] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:42 2022] DELETE /identity/v3/projects/f19d52bf09ee45718b8368f0d5c163ad => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:43.080723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:43.082124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:43.094646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3b858-5238-47f7-827e-2eda7191f901 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:43.150985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:43.151834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:43.151946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:43.152261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] PATH_INFO: `/v3/users/c422798ac6ca447195307b1e5db6bce0` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:43.153220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c422798ac6ca447195307b1e5db6bce0)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:43.155171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:43.206113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c422798ac6ca447195307b1e5db6bce0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.213642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c422798ac6ca447195307b1e5db6bce0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.228281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c422798ac6ca447195307b1e5db6bce0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.234320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c422798ac6ca447195307b1e5db6bce0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.242927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d995555-ea5a-4657-8a8e-954640a5abbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c422798ac6ca447195307b1e5db6bce0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.249624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1130/2260] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:43 2022] DELETE /identity/v3/users/c422798ac6ca447195307b1e5db6bce0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:43.563900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:43.565571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:43.581547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8735a4-2caf-432f-a118-ebaccae9008f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:43.665443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:43.666559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:43.666990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:43.667164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] PATH_INFO: `/v3/projects/c24a2159b30040b28596278233d87a9d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:43.669082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24a2159b30040b28596278233d87a9d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:43.671225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:43.726446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24a2159b30040b28596278233d87a9d', 'request_id': 'req-02330a98-2f2e-475a-b3f0-84ecebc32319'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.732047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24a2159b30040b28596278233d87a9d', 'request_id': 'req-02330a98-2f2e-475a-b3f0-84ecebc32319'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.739072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24a2159b30040b28596278233d87a9d', 'request_id': 'req-02330a98-2f2e-475a-b3f0-84ecebc32319'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.753226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] The token cache is being invalidate because project c24a2159b30040b28596278233d87a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:43.753521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.754207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02330a98-2f2e-475a-b3f0-84ecebc32319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:43.755751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1131/2261] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:43 2022] DELETE /identity/v3/projects/c24a2159b30040b28596278233d87a9d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:43.761924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:43.762415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:43.762512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:43.783406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:43.784630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:43.794314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:43.845283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1131/2262] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:43.854551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:43.854840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:43.855123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:43.875501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:43.876703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:43.884568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1132/2263] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:43.888688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:43.898621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbe83a-0480-4323-be16-f10be36139cd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:43.903371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:43.904109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:43.904291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:43.904579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:43.905211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:43.905545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:43.907733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:43.921822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1132/2264] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:43.923971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:43.934474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817cbf5-6660-4f1d-985b-5651b2e1b822 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:43.939744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:43.940463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:43.940811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:43.941169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:43.941923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:43.943853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:43.959710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:43.975490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1133/2265] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:43 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:43.979619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:43.990108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223f436-b112-45b9-8e11-c97643f63226 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.051736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.052350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.052632 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.052907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.053543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.055491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.077307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1133/2266] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:43 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:44.082198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.093708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6590b-0972-4e13-89a3-f9fe74564434 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.098594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.099194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.099472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.099745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.100389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.101994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.112470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1134/2267] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:44.116824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.127257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4149b-2430-47c1-81e2-135704a3cd92 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.131770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c19e4-77a8-4d05-9946-85ea8b6da3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.132352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c19e4-77a8-4d05-9946-85ea8b6da3da admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.132630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c19e4-77a8-4d05-9946-85ea8b6da3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.132899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c19e4-77a8-4d05-9946-85ea8b6da3da admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.133523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c19e4-77a8-4d05-9946-85ea8b6da3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.135140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c19e4-77a8-4d05-9946-85ea8b6da3da admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.145421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1134/2268] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:44.149379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.159145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0edd47-9667-4522-865a-9dabcbd34a07 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.163726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.164395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.164676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.164948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] PATH_INFO: `/v3/projects/43188ff808244e749fa408120b1207fd/users/18c5c7c92c9a4d3d9b59dc6fd697d2d8/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.179890 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43188ff808244e749fa408120b1207fd, user_id=18c5c7c92c9a4d3d9b59dc6fd697d2d8, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.182663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.195432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1135/2269] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:44 2022] PUT /identity/v3/projects/43188ff808244e749fa408120b1207fd/users/18c5c7c92c9a4d3d9b59dc6fd697d2d8/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:44.206645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.206934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.207205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.233651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] MFA Rules not processed for user `18c5c7c92c9a4d3d9b59dc6fd697d2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:44.235334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:44.297721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1135/2270] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:44.305675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.369915 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af36ae-ed0b-441d-bd57-fde2c7b5b74c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.377634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.378259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.378540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.378812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.379434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.381304 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.407707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1136/2271] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:44 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:44.411598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.422840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588f9ca-3335-4602-805b-4db8b8e6ee04 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.431032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:44.480965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.481566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.481844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.482140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.482762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.485257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.507074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1136/2272] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:44 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:44.511066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.521207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eba04-3764-4f01-a7c2-140a241fd095 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.525814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.526429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.526714 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.526986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.527624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.529225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.540307 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1137/2273] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:01:44.544394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.554764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ab484-3e74-403c-b188-60f57c1cbd60 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.559364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.560003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.560286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.560564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] PATH_INFO: `/v3/projects/d8e6282d56c14b5896718b9a7c58f29f/users/3b4f9be1b40f4ec98ee2c221f4683e43/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.574995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e6282d56c14b5896718b9a7c58f29f, user_id=3b4f9be1b40f4ec98ee2c221f4683e43, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.577971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.590553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1137/2274] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:44 2022] PUT /identity/v3/projects/d8e6282d56c14b5896718b9a7c58f29f/users/3b4f9be1b40f4ec98ee2c221f4683e43/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:44.594981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.604678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43901bf-ff92-4b95-b90c-25555888cb8e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.662827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317885c5-952b-45e2-b2b4-1b098447b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.663452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317885c5-952b-45e2-b2b4-1b098447b939 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.663743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317885c5-952b-45e2-b2b4-1b098447b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.664024 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317885c5-952b-45e2-b2b4-1b098447b939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.664676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317885c5-952b-45e2-b2b4-1b098447b939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.666313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317885c5-952b-45e2-b2b4-1b098447b939 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.677968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1138/2275] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:44.682586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.693110 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d394d-ab53-4ce8-a06f-91d6ecf8cf39 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.698232 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.698942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.699261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.700153 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] PATH_INFO: `/v3/projects/d8e6282d56c14b5896718b9a7c58f29f/users/3b4f9be1b40f4ec98ee2c221f4683e43/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.702379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e6282d56c14b5896718b9a7c58f29f, user_id=3b4f9be1b40f4ec98ee2c221f4683e43, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.705430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.721751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1138/2276] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:44 2022] PUT /identity/v3/projects/d8e6282d56c14b5896718b9a7c58f29f/users/3b4f9be1b40f4ec98ee2c221f4683e43/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:44.732762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.733051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.733329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.761513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] MFA Rules not processed for user `3b4f9be1b40f4ec98ee2c221f4683e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:44.762690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:44.779037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:44.832176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1139/2277] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:44 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:44.842478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:44.844787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:44.904086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c4569-91f1-467a-aaa4-e0f2804407a2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:44.908554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:44.909159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:44.909438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:44.909708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:44.919789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:44.921227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:44.931886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1139/2278] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:44 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:45.012796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:45.022814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ef9c-961a-4b8a-8974-17288d49e128 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:45.026368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fad26-c897-4d85-9464-02801f5da13c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:45.026958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fad26-c897-4d85-9464-02801f5da13c service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:45.027262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fad26-c897-4d85-9464-02801f5da13c service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:45.027537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fad26-c897-4d85-9464-02801f5da13c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:45.085797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fad26-c897-4d85-9464-02801f5da13c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:45.087376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fad26-c897-4d85-9464-02801f5da13c service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:45.096429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1140/2279] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:45 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:50.403574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:50.480817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1e49e-13db-4202-a22c-98bddf7ba80d service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:50.486021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:50.486672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:50.486987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:50.487235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] PATH_INFO: `/v3/users/18c5c7c92c9a4d3d9b59dc6fd697d2d8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:50.489028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c5c7c92c9a4d3d9b59dc6fd697d2d8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:50.491044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:50.499033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:50.551696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c5c7c92c9a4d3d9b59dc6fd697d2d8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:50.561518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c5c7c92c9a4d3d9b59dc6fd697d2d8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:50.577835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c5c7c92c9a4d3d9b59dc6fd697d2d8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:50.579142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c5c7c92c9a4d3d9b59dc6fd697d2d8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:50.586288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c5c7c92c9a4d3d9b59dc6fd697d2d8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:50.592297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1140/2280] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:50 2022] DELETE /identity/v3/users/18c5c7c92c9a4d3d9b59dc6fd697d2d8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:50.597026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:50.597521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:50.597620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:50.626448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:50.627711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:50.696134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1141/2281] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:50.724634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:50.726154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:50.841071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057106aa-7ca4-4140-8578-04723f2cd3d5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:50.849456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb2a7-7ceb-4b08-9a10-39af62c06845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:50.854975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb2a7-7ceb-4b08-9a10-39af62c06845 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:50.854975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb2a7-7ceb-4b08-9a10-39af62c06845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:50.854975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb2a7-7ceb-4b08-9a10-39af62c06845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:50.864931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb2a7-7ceb-4b08-9a10-39af62c06845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:50.867230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cb2a7-7ceb-4b08-9a10-39af62c06845 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:50.879080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1141/2282] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:51.233377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:51.234995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:51.248358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637b31f-05d7-48dd-a429-b9f6456bcd98 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:51.273833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:51.274749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:51.275180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:51.275555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] PATH_INFO: `/v3/projects/43188ff808244e749fa408120b1207fd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:51.282346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43188ff808244e749fa408120b1207fd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:51.284267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:51.339031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:51.339761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:51.340062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:51.352904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:51.358553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43188ff808244e749fa408120b1207fd', 'request_id': 'req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.372005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43188ff808244e749fa408120b1207fd', 'request_id': 'req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.392392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43188ff808244e749fa408120b1207fd', 'request_id': 'req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.399112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] MFA Rules not processed for user `9f74f51060554db18bc96199222e9261`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:51.400557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:51.407551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] The token cache is being invalidate because project 43188ff808244e749fa408120b1207fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:51.408095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.408979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.411658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1142/2283] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:51 2022] DELETE /identity/v3/projects/43188ff808244e749fa408120b1207fd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:51.415492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:51.416948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:51.511287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1142/2284] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:01:51.533738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5948c-b675-40ee-9ab7-7b7e544ef4f0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:51.539762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:51.540406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:51.540692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:51.540968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] PATH_INFO: `/v3/users/3b4f9be1b40f4ec98ee2c221f4683e43` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:51.542550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b4f9be1b40f4ec98ee2c221f4683e43)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:51.544346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:51.617751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b4f9be1b40f4ec98ee2c221f4683e43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.627400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b4f9be1b40f4ec98ee2c221f4683e43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.645625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b4f9be1b40f4ec98ee2c221f4683e43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.647163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b4f9be1b40f4ec98ee2c221f4683e43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.660533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f84980d6-9ea6-41b5-8f5a-bce8888bea26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b4f9be1b40f4ec98ee2c221f4683e43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:51.672352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1143/2285] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:51 2022] DELETE /identity/v3/users/3b4f9be1b40f4ec98ee2c221f4683e43 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:52.108335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:52.130782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff919796-1629-4294-bb99-ae36f1b10925 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:52.207875 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:52.256047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:52.258092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:52.258419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:52.258700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] PATH_INFO: `/v3/projects/d8e6282d56c14b5896718b9a7c58f29f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:52.260128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e6282d56c14b5896718b9a7c58f29f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:52.263833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:52.394334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e6282d56c14b5896718b9a7c58f29f', 'request_id': 'req-f4a8f7fa-47d5-4704-beec-81b73e41d711'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:52.414814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e6282d56c14b5896718b9a7c58f29f', 'request_id': 'req-f4a8f7fa-47d5-4704-beec-81b73e41d711'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:52.439704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e6282d56c14b5896718b9a7c58f29f', 'request_id': 'req-f4a8f7fa-47d5-4704-beec-81b73e41d711'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:52.475245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] The token cache is being invalidate because project d8e6282d56c14b5896718b9a7c58f29f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:52.476050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:52.477320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f4a8f7fa-47d5-4704-beec-81b73e41d711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:52.481228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1143/2286] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:52 2022] DELETE /identity/v3/projects/d8e6282d56c14b5896718b9a7c58f29f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:52.490174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:52.491201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:52.491888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:52.546522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:52.548495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:52.640973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:52.660547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1144/2287] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:52.668536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:52.669106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:52.669264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:52.698959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:52.701281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:52.719538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1144/2288] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:52.720723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:52.752475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df7998-1b3e-4f99-aec1-a263911b8b8a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:52.765702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:52.767897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:52.768813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:52.769248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:52.770790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:01:52.771258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:52.773354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:52.793237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1145/2289] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:01:52.798147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:52.810099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636eb7d-92d3-4d8f-85f5-a21a1447e2c6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:52.819295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:52.820274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:52.820558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:52.820894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:52.821564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:52.823852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:52.873937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1145/2290] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:52 2022] POST /identity/v3/projects => generated 389 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:52.876713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:52.891280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa3eb8-bc27-4a0a-89a8-9d717fcb9b60 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.023480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.024528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.025181 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.025639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.026564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.029177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.077749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1146/2291] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:52 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:53.079140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.097111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb49f9-5a06-4bd1-a1d0-08193c004624 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.106947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.108036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.108469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.110295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.111222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.113433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.128553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1146/2292] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:53.129754 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.142481 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626b671-5bb1-4015-8c50-efada7ded44c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.148606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3769d-0921-40ad-a549-167aca3d927c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.149212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3769d-0921-40ad-a549-167aca3d927c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.149499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3769d-0921-40ad-a549-167aca3d927c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.149773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3769d-0921-40ad-a549-167aca3d927c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.150458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3769d-0921-40ad-a549-167aca3d927c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.152060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3769d-0921-40ad-a549-167aca3d927c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.168410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1147/2293] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:53.170699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.185569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5515b-ac66-4be9-84fd-4576436db9f2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.195401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.196276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.196655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.197031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] PATH_INFO: `/v3/projects/bd937209a2cb4ffc88f8338cbf425e07/users/6fe7b26f4f3d4c25ac66e1585cc3ab31/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.223899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd937209a2cb4ffc88f8338cbf425e07, user_id=6fe7b26f4f3d4c25ac66e1585cc3ab31, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.225856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.246252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1147/2294] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:53 2022] PUT /identity/v3/projects/bd937209a2cb4ffc88f8338cbf425e07/users/6fe7b26f4f3d4c25ac66e1585cc3ab31/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:53.256683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.257123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.257470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.301191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] MFA Rules not processed for user `6fe7b26f4f3d4c25ac66e1585cc3ab31`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:53.302878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:53.380486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1148/2295] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:53.384949 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.461773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd6e26-5700-4143-8de9-a26686d05cd9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.466502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.467112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.467393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.467660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.468442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.470975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.477230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:53.508233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1148/2296] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:01:53 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:53.513164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.531729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93982a30-16ed-4201-8c49-aab6ec3b5c8f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.610813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:53.624216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.625365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.625650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.626273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.626573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.628492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.658097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1149/2297] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:01:53 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:01:53.659893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.670452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f27e2-4133-4c2d-98ce-5ebd7b8b5139 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.676513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.677148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.677576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.677871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.678539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.680444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.694613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1149/2298] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:53.698921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.712273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dc3cd-37c0-4c0e-b245-ac4d15be6ed2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.716768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.717392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.717671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.717972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] PATH_INFO: `/v3/projects/ac05a78525de4c00a45f930a0eda3637/users/b6586a0b633f40819ca82cec8f659c4e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.733023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac05a78525de4c00a45f930a0eda3637, user_id=b6586a0b633f40819ca82cec8f659c4e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.736656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.750976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1150/2299] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:53 2022] PUT /identity/v3/projects/ac05a78525de4c00a45f930a0eda3637/users/b6586a0b633f40819ca82cec8f659c4e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:53.755057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.765069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fd039-6f7b-4bf6-9244-6e28d82b83d9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.830839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c6752-1371-4425-bb17-6c97b945c830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.831461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c6752-1371-4425-bb17-6c97b945c830 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.831739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c6752-1371-4425-bb17-6c97b945c830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.832008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c6752-1371-4425-bb17-6c97b945c830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.832706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c6752-1371-4425-bb17-6c97b945c830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.834345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c6752-1371-4425-bb17-6c97b945c830 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.845126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1150/2300] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:01:53.849266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:53.860511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07856c8a-e9ea-42c0-8371-b7404e9773ab admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:53.865505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:53.866228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.866511 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.866788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] PATH_INFO: `/v3/projects/ac05a78525de4c00a45f930a0eda3637/users/b6586a0b633f40819ca82cec8f659c4e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.868668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac05a78525de4c00a45f930a0eda3637, user_id=b6586a0b633f40819ca82cec8f659c4e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:53.871359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:53.882418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1151/2301] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:01:53 2022] PUT /identity/v3/projects/ac05a78525de4c00a45f930a0eda3637/users/b6586a0b633f40819ca82cec8f659c4e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:53.949275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:53.949555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:53.949824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:53.982096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] MFA Rules not processed for user `b6586a0b633f40819ca82cec8f659c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:53.983451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:54.051738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1151/2302] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:54.067493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:54.069362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:54.131274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36783f1b-1fed-4be1-b1c0-f39f0b58eb4a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:54.134955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198031f-6d82-4164-b3fb-5986082d41fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:54.135566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198031f-6d82-4164-b3fb-5986082d41fb service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:54.135837 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198031f-6d82-4164-b3fb-5986082d41fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:54.136122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198031f-6d82-4164-b3fb-5986082d41fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:54.145502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198031f-6d82-4164-b3fb-5986082d41fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:54.146949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198031f-6d82-4164-b3fb-5986082d41fb service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:54.156788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1152/2303] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:54.294795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:54.355495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:54.362588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bbca7-bce8-43d0-b319-dd4c3c502472 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:54.368072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:54.368707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:54.368989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:54.369260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] PATH_INFO: `/v3/users/6fe7b26f4f3d4c25ac66e1585cc3ab31` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:54.370413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe7b26f4f3d4c25ac66e1585cc3ab31)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:54.372336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:54.418055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe7b26f4f3d4c25ac66e1585cc3ab31'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:54.424933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe7b26f4f3d4c25ac66e1585cc3ab31'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:54.436535 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe7b26f4f3d4c25ac66e1585cc3ab31'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:54.437751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe7b26f4f3d4c25ac66e1585cc3ab31'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:54.444858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe7b26f4f3d4c25ac66e1585cc3ab31'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:54.451394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1152/2304] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:54 2022] DELETE /identity/v3/users/6fe7b26f4f3d4c25ac66e1585cc3ab31 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:54.456420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:54.456706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:54.457009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:54.477735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:54.479194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:54.538587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1153/2305] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:01:54.548066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:54.549473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:54.608226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a316-c001-4fdb-bb3b-6bac1f3d60bd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:54.612338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:54.612937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:54.613214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:54.613483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:54.625697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:54.626948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:54.638371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1153/2306] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:01:54.995858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:54.997354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:55.009700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3954d-75c0-4c71-9588-c262d9874c6b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:55.014341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:55.015043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:55.015447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:55.016288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] PATH_INFO: `/v3/projects/bd937209a2cb4ffc88f8338cbf425e07` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:55.017497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd937209a2cb4ffc88f8338cbf425e07)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:55.019606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:55.060599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:55.070173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd937209a2cb4ffc88f8338cbf425e07', 'request_id': 'req-4b9c3324-549f-4b80-a1db-250d4f08f8b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.079591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd937209a2cb4ffc88f8338cbf425e07', 'request_id': 'req-4b9c3324-549f-4b80-a1db-250d4f08f8b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.087434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd937209a2cb4ffc88f8338cbf425e07', 'request_id': 'req-4b9c3324-549f-4b80-a1db-250d4f08f8b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.107103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] The token cache is being invalidate because project bd937209a2cb4ffc88f8338cbf425e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:55.107406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.108347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.111103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1154/2307] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:54 2022] DELETE /identity/v3/projects/bd937209a2cb4ffc88f8338cbf425e07 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:55.115018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:55.133169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b302f6d-da24-4ad7-82e4-2d9b89fe125c service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:55.202010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:55.202648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:55.203500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:55.203782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] PATH_INFO: `/v3/users/b6586a0b633f40819ca82cec8f659c4e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:55.204930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6586a0b633f40819ca82cec8f659c4e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:55.206903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:55.227218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:01:55.252348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6586a0b633f40819ca82cec8f659c4e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.259476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6586a0b633f40819ca82cec8f659c4e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.271700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6586a0b633f40819ca82cec8f659c4e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.272323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6586a0b633f40819ca82cec8f659c4e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.279055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c48c14ec-7818-4edf-bac7-4f61c3487ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6586a0b633f40819ca82cec8f659c4e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.284960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1154/2308] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:01:55 2022] DELETE /identity/v3/users/b6586a0b633f40819ca82cec8f659c4e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:01:55.574783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:01:55.576496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:55.593103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3324-549f-4b80-a1db-250d4f08f8b2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:01:55.665540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:01:55.666117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:55.666401 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:55.666674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] PATH_INFO: `/v3/projects/ac05a78525de4c00a45f930a0eda3637` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:55.667782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac05a78525de4c00a45f930a0eda3637)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:01:55.669677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:01:55.723043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac05a78525de4c00a45f930a0eda3637', 'request_id': 'req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.730087 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac05a78525de4c00a45f930a0eda3637', 'request_id': 'req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.739221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac05a78525de4c00a45f930a0eda3637', 'request_id': 'req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.763068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] The token cache is being invalidate because project ac05a78525de4c00a45f930a0eda3637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:01:55.763643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.764433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ccfec34e-5ce9-4a02-a2cb-ce6e7a4c68fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:01:55.773960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1155/2309] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:01:55 2022] DELETE /identity/v3/projects/ac05a78525de4c00a45f930a0eda3637 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:01:58.743274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:01:58.743752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:01:58.744132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:01:58.766486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] MFA Rules not processed for user `db5a0bddb68047d68f0c2dd1af0cd57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:01:58.767757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:01:58.829979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1155/2310] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:01:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:04.793113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca50603-1eb3-411c-9b43-4ada20ba6313 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:04.793658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca50603-1eb3-411c-9b43-4ada20ba6313 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:04.793762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca50603-1eb3-411c-9b43-4ada20ba6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:04.813671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3ca50603-1eb3-411c-9b43-4ada20ba6313 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:04.814914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3ca50603-1eb3-411c-9b43-4ada20ba6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:04.873313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1156/2311] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:02:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:04.885639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:04.887132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:04.967531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e0155-81ec-41ce-9b79-1526b2714074 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:04.973229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:04.973840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:04.974144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:04.974421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:04.985521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:04.987164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:04.995806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1156/2312] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:02:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 15:02:14.866748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:14.992443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=8,keystone:SELECT=77,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:17.357986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdaf39-73cb-4b01-8340-8213d9c2dbc9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:17.358578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdaf39-73cb-4b01-8340-8213d9c2dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:17.358578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdaf39-73cb-4b01-8340-8213d9c2dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:17.384934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5ccdaf39-73cb-4b01-8340-8213d9c2dbc9 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:17.396845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5ccdaf39-73cb-4b01-8340-8213d9c2dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:17.426580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1157/2313] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:02:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:17.440295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:17.450831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64916f1-995f-4b79-b0b4-9ab43bf656e2 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:17.456350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:17.457230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:17.457838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:17.458311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:17.467798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:17.471022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:17.482987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1157/2314] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:02:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:02:27.376018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:27.478552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:34.157655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254c5d-f23b-4e91-807b-66d8f689cb8a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:34.158190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254c5d-f23b-4e91-807b-66d8f689cb8a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:34.158331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96254c5d-f23b-4e91-807b-66d8f689cb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:34.179294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-96254c5d-f23b-4e91-807b-66d8f689cb8a None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:34.180482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96254c5d-f23b-4e91-807b-66d8f689cb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:34.188983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1158/2315] 10.222.0.61 () {58 vars in 967 bytes} [Sat Sep 24 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:34.201241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:34.214395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1195645-b279-4351-b3d3-83a0d14b5f61 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:34.220161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44905b0-448d-4f78-82bb-88153e2ae493 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:34.220965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44905b0-448d-4f78-82bb-88153e2ae493 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:34.221170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44905b0-448d-4f78-82bb-88153e2ae493 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:34.221492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44905b0-448d-4f78-82bb-88153e2ae493 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:34.230964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44905b0-448d-4f78-82bb-88153e2ae493 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:34.232584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44905b0-448d-4f78-82bb-88153e2ae493 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:34.241938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1158/2316] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:02:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:02:36.302569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad75c51-4352-4d34-a81d-09c8f4c62c92 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:36.304060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad75c51-4352-4d34-a81d-09c8f4c62c92 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:36.304501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad75c51-4352-4d34-a81d-09c8f4c62c92 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:36.306453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1159/2317] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:02:36.333893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:36.334526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:36.334894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:36.357765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] MFA Rules not processed for user `359a86b84b994530b005b43c6c711884`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:36.361079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:36.378612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad75c51-4352-4d34-a81d-09c8f4c62c92 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:36.380694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aad75c51-4352-4d34-a81d-09c8f4c62c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:36.443443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1159/2318] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:36.455780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:36.457624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:36.474622 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc018c4-f7a0-4163-9f92-0b047b83d4ab None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:36.480742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:36.481358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:36.481790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:36.482237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:36.486604 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad75c51-4352-4d34-a81d-09c8f4c62c92 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:36.492335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:36.493947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:36.494809 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:36.495531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:36.495932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:36.496789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] PATH_INFO: `/v3/users/67f6f1b1653246fb82e9d48813db8701` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:36.498233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67f6f1b1653246fb82e9d48813db8701)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:36.500893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:36.506684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1160/2319] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:02:36.581651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67f6f1b1653246fb82e9d48813db8701'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:36.590996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67f6f1b1653246fb82e9d48813db8701'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:36.606315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67f6f1b1653246fb82e9d48813db8701'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:36.607293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67f6f1b1653246fb82e9d48813db8701'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:36.618482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2e536a98-7271-4385-a138-b429cd50ac83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67f6f1b1653246fb82e9d48813db8701'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:36.625135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1160/2320] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:02:36 2022] DELETE /identity/v3/users/67f6f1b1653246fb82e9d48813db8701 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:36.876546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:36.907899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465504b-3669-4c1a-ada0-33eb5471a61a service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:36.991383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:36.992018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:36.992341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:36.992615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] PATH_INFO: `/v3/projects/934382300f5f4af98706b721ce84ae36` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:36.993768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934382300f5f4af98706b721ce84ae36)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:36.995863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.045310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934382300f5f4af98706b721ce84ae36', 'request_id': 'req-28dfb458-4ccf-4042-a6a4-90e640260162'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.051537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934382300f5f4af98706b721ce84ae36', 'request_id': 'req-28dfb458-4ccf-4042-a6a4-90e640260162'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.058600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934382300f5f4af98706b721ce84ae36', 'request_id': 'req-28dfb458-4ccf-4042-a6a4-90e640260162'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.070462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:37.076175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] The token cache is being invalidate because project 934382300f5f4af98706b721ce84ae36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:02:37.076885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.077837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-28dfb458-4ccf-4042-a6a4-90e640260162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.081129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1161/2321] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:02:36 2022] DELETE /identity/v3/projects/934382300f5f4af98706b721ce84ae36 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:37.086075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.086373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.086647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.107021 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:37.108306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:37.169682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1161/2322] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:37.177442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.177716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.178024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.198967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:37.200189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:37.208977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1162/2323] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:37.213423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.223559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6059520-95c6-4f37-9ec9-628720c52511 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.228223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.228823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.229100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.229373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.230077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:02:37.230382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.231970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.240866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:37.244272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1162/2324] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:02:37.248631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.260648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54e2be-9fb0-4069-b73b-2f382108ac93 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.268063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.268696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.268978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.269448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.270612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.273043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.306127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1163/2325] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:02:37 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:02:37.311781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.322448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a00174-5ba0-4b06-8724-13b92c61dc97 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.378884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.379498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.379780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.380053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.380691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.382671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.405012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1163/2326] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:02:37 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:02:37.409255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.419859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f688187-e61f-4df5-81e9-383ab00a693a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.425027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.425617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.425920 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.426208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.426880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.428469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.444616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1164/2327] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:02:37.450866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.461436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a2d3-9aad-47ae-bcdc-569ff05fea28 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.468112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3878e-5567-4b09-a277-6a8ccb55bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.468727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3878e-5567-4b09-a277-6a8ccb55bc73 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.469083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3878e-5567-4b09-a277-6a8ccb55bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.469370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3878e-5567-4b09-a277-6a8ccb55bc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.470148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3878e-5567-4b09-a277-6a8ccb55bc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.471803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3878e-5567-4b09-a277-6a8ccb55bc73 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.482910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1164/2328] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:02:37.487903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.500736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f1d0-43b1-4b95-a2bb-4242afc7eef5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.506317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.507298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.507589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.507866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] PATH_INFO: `/v3/projects/72cc264568154da38f2b0162b84ad04b/users/76e9fe3579274f4da86f4790c00da79b/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.523559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72cc264568154da38f2b0162b84ad04b, user_id=76e9fe3579274f4da86f4790c00da79b, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.526386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.540548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1165/2329] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:02:37 2022] PUT /identity/v3/projects/72cc264568154da38f2b0162b84ad04b/users/76e9fe3579274f4da86f4790c00da79b/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:02:37.551028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.551316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.551595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.579120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] MFA Rules not processed for user `76e9fe3579274f4da86f4790c00da79b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:37.580243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:37.644925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1165/2330] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:37.656531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.658255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:37.713482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a8fb-80cf-4cd5-8f87-c0b6110bbb40 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.717115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd95c1-7f7e-4f19-80ee-5d5b64833390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.717744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd95c1-7f7e-4f19-80ee-5d5b64833390 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.718058 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd95c1-7f7e-4f19-80ee-5d5b64833390 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.718332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd95c1-7f7e-4f19-80ee-5d5b64833390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.727801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd95c1-7f7e-4f19-80ee-5d5b64833390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.729233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd95c1-7f7e-4f19-80ee-5d5b64833390 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.738533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1166/2331] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:37.763360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:37.825116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580a3f-bbaa-4102-b579-961742632b39 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:37.829795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:37.830432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.830743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.831016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] PATH_INFO: `/v3/users/76e9fe3579274f4da86f4790c00da79b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.833103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e9fe3579274f4da86f4790c00da79b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:37.834673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:37.858725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:37.901174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e9fe3579274f4da86f4790c00da79b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.914855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e9fe3579274f4da86f4790c00da79b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.933418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e9fe3579274f4da86f4790c00da79b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.934935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e9fe3579274f4da86f4790c00da79b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.943477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e9fe3579274f4da86f4790c00da79b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:37.950747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1166/2332] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:02:37 2022] DELETE /identity/v3/users/76e9fe3579274f4da86f4790c00da79b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:37.954998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:37.955292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:37.955902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:37.985988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:37.987400 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:38.099422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1167/2333] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:38.111380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:38.113121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:38.217234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee09c9-d0b2-469a-80b3-03b8d0110f2e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:38.220993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a219243-c731-44cd-97f4-bb9c56f7a297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:38.221612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a219243-c731-44cd-97f4-bb9c56f7a297 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.221864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a219243-c731-44cd-97f4-bb9c56f7a297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.222198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a219243-c731-44cd-97f4-bb9c56f7a297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.233846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a219243-c731-44cd-97f4-bb9c56f7a297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:38.235971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a219243-c731-44cd-97f4-bb9c56f7a297 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:38.249232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1167/2334] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:38.636284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:38.639017 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:38.657650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e499e0-4682-4155-8bce-98faa6e97c0b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:38.664984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:38.665821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.666691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.667057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] PATH_INFO: `/v3/projects/72cc264568154da38f2b0162b84ad04b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.668557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=72cc264568154da38f2b0162b84ad04b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:38.673591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:38.720330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:38.733063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72cc264568154da38f2b0162b84ad04b', 'request_id': 'req-bc75c3be-2b35-4e51-a398-911ab30f3b4b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:38.739744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72cc264568154da38f2b0162b84ad04b', 'request_id': 'req-bc75c3be-2b35-4e51-a398-911ab30f3b4b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:38.746152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72cc264568154da38f2b0162b84ad04b', 'request_id': 'req-bc75c3be-2b35-4e51-a398-911ab30f3b4b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:38.759317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] The token cache is being invalidate because project 72cc264568154da38f2b0162b84ad04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:02:38.759610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:38.760234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bc75c3be-2b35-4e51-a398-911ab30f3b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:38.761883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1168/2335] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:02:38 2022] DELETE /identity/v3/projects/72cc264568154da38f2b0162b84ad04b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:38.767730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.768229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.768440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.789978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:38.791188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:38.854129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1168/2336] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:02:38.862101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.862322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.862598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.883345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:38.884722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:38.893863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1169/2337] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:38.898708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:38.908893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf12764-0bc2-42b0-b49e-970bc6113006 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:38.914790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:38.915686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.915964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.916236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.916927 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:02:38.917233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:38.918873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e207f-3414-49c3-a986-3360711cbedf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:38.932454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1169/2338] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:02:38.937104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:38.954422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cd8f0-df9e-4b5d-a6c8-bdcf693eac00 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:38.955960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932a5a3-8020-4ae9-a07d-887b6d35b080 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.956247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932a5a3-8020-4ae9-a07d-887b6d35b080 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.956547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932a5a3-8020-4ae9-a07d-887b6d35b080 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.958108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1170/2339] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:02:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:02:38.962874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:38.963491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.963776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.964079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.964758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:38.964871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:38.965212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:38.965621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:38.968231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:38.993358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:38.996373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.005073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1170/2340] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:02:38 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:02:39.008537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.035953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:39.109650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1e219-70c8-4f52-be98-093e887dd85e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.117540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.118154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.118579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.119102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.119945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.122934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.125664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1171/2341] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:02:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:39.142583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.144775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.152152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1171/2342] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:02:39 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:02:39.154926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.175086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9b1ab-6f49-401e-97c1-f504a8ff5066 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.182872 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.183725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.184289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.184771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.186181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.188587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.204588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1172/2343] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:02:39.208202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.219876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6296e50-2207-4efd-a985-891a11dde850 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.224880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.225775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.227064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.227425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.228259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.229954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.247887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1173/2344] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:02:39.251304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.257353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3462a-a56c-43c4-a8ac-74519613cd44 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.265379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131259-e6b4-43d4-8dd3-ab587d30ae47 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.276285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e76e7a-eef8-4953-b5e0-881004a87c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.278889 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e76e7a-eef8-4953-b5e0-881004a87c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.279856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e76e7a-eef8-4953-b5e0-881004a87c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.280193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e76e7a-eef8-4953-b5e0-881004a87c4c admin admin] PATH_INFO: `/v3/projects/43e1ec63be6740fa9516a2e5331986a2/users/c8a3f8bef9f0431eac090f8a520f744b/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.298370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e76e7a-eef8-4953-b5e0-881004a87c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e1ec63be6740fa9516a2e5331986a2, user_id=c8a3f8bef9f0431eac090f8a520f744b, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.301553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e76e7a-eef8-4953-b5e0-881004a87c4c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.319303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1174/2345] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:02:39 2022] PUT /identity/v3/projects/43e1ec63be6740fa9516a2e5331986a2/users/c8a3f8bef9f0431eac090f8a520f744b/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:39.324578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.324578 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.324908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.356858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b93be2-b918-40df-9575-4b9ee6735b6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.357601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b93be2-b918-40df-9575-4b9ee6735b6e service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.357983 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b93be2-b918-40df-9575-4b9ee6735b6e service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.358329 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b93be2-b918-40df-9575-4b9ee6735b6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.367333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:39.368721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.405833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:39.419542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:39.436385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b93be2-b918-40df-9575-4b9ee6735b6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.437938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b93be2-b918-40df-9575-4b9ee6735b6e service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.447983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1175/2346] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:02:39.449210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1172/2347] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:39.462366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.463758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.524735 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b0210-dda4-4748-833f-f4e9e537b3d7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.525184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b0210-dda4-4748-833f-f4e9e537b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.525465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b0210-dda4-4748-833f-f4e9e537b3d7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.526879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1173/2348] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 15:02:39.527332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c39979-7e6c-48ef-8e34-aa414772ff62 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.530935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.532055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.532175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f321c-1cf9-4af1-beb1-bee3f55bc8bf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.532333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.532431 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f321c-1cf9-4af1-beb1-bee3f55bc8bf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.532599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.532699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f321c-1cf9-4af1-beb1-bee3f55bc8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.541511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.542962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.552341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-615f321c-1cf9-4af1-beb1-bee3f55bc8bf None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:39.553052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1176/2349] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:39.553545 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-615f321c-1cf9-4af1-beb1-bee3f55bc8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.560079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1174/2350] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:02:39.571808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.573398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.586434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb210d6-5e7a-4cd9-8a7f-d857ead6da3a service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.590144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecd17a-6e70-45e9-b52d-0452efec2007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.590740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecd17a-6e70-45e9-b52d-0452efec2007 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.591022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecd17a-6e70-45e9-b52d-0452efec2007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.591298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecd17a-6e70-45e9-b52d-0452efec2007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.599405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecd17a-6e70-45e9-b52d-0452efec2007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.600837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecd17a-6e70-45e9-b52d-0452efec2007 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.609039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1177/2351] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:39.698879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35214eb9-ded4-4d8f-9b4e-e6300800b75d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.699184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35214eb9-ded4-4d8f-9b4e-e6300800b75d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.699594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35214eb9-ded4-4d8f-9b4e-e6300800b75d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.702227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1175/2352] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:02:39.710520 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.710943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.711075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.736130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:39.737780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:39.828930 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1178/2353] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:39.840251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35214eb9-ded4-4d8f-9b4e-e6300800b75d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:39.852212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35214eb9-ded4-4d8f-9b4e-e6300800b75d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:39.856090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003471c-6cd5-4aa2-85c0-1693be6cc6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:39.856788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003471c-6cd5-4aa2-85c0-1693be6cc6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:39.857165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003471c-6cd5-4aa2-85c0-1693be6cc6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:39.857535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003471c-6cd5-4aa2-85c0-1693be6cc6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:39.871123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003471c-6cd5-4aa2-85c0-1693be6cc6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:39.873393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003471c-6cd5-4aa2-85c0-1693be6cc6d8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:39.889755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1176/2354] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:43.261167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:43.262996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:43.282745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f65a-5e38-4d6a-abe1-b3f2f25c7486 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:43.288290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:43.288886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:43.289171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:43.289452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] PATH_INFO: `/v3/users/3d79d6bc06124a6f93904fb6e5743ade` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:43.290764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d79d6bc06124a6f93904fb6e5743ade)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:43.292799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:43.355408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d79d6bc06124a6f93904fb6e5743ade'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:43.372433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d79d6bc06124a6f93904fb6e5743ade'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:43.373989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:43.393705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d79d6bc06124a6f93904fb6e5743ade'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:43.394736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d79d6bc06124a6f93904fb6e5743ade'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:43.405093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d79d6bc06124a6f93904fb6e5743ade'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:43.418412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1179/2355] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:02:43 2022] DELETE /identity/v3/users/3d79d6bc06124a6f93904fb6e5743ade => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:43.423112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:43.424092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:43.424560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:43.465815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:43.467476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:43.554653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1177/2356] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:43.562891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:43.565248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:43.638744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4accee-afe0-4850-a975-c4dd48296475 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:43.642636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d4b2d-2137-42df-a950-e33add16bae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:43.643257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d4b2d-2137-42df-a950-e33add16bae4 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:43.643542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d4b2d-2137-42df-a950-e33add16bae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:43.643817 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d4b2d-2137-42df-a950-e33add16bae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:43.654109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d4b2d-2137-42df-a950-e33add16bae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:43.655574 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d4b2d-2137-42df-a950-e33add16bae4 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:43.666129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1180/2357] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:44.039205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:44.040978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:44.064033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45896d8d-8918-45a4-96c6-27dc35f81081 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:44.082726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:44.084971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:44.085370 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:44.085778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] PATH_INFO: `/v3/projects/cb11a698923049a9bc314419acee7466` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:44.089296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb11a698923049a9bc314419acee7466)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:44.092662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:44.208115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb11a698923049a9bc314419acee7466', 'request_id': 'req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:44.217526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb11a698923049a9bc314419acee7466', 'request_id': 'req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:44.227471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb11a698923049a9bc314419acee7466', 'request_id': 'req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:44.250824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] The token cache is being invalidate because project cb11a698923049a9bc314419acee7466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:02:44.251135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:44.253495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:02:44.258263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1178/2358] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:02:44 2022] DELETE /identity/v3/projects/cb11a698923049a9bc314419acee7466 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:45.431991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7edcc5-0e5e-4d3c-b35d-27953129f609 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.432540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7edcc5-0e5e-4d3c-b35d-27953129f609 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.432649 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7edcc5-0e5e-4d3c-b35d-27953129f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.473363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-df7edcc5-0e5e-4d3c-b35d-27953129f609 None None] MFA Rules not processed for user `c8a3f8bef9f0431eac090f8a520f744b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:45.474722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-df7edcc5-0e5e-4d3c-b35d-27953129f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:45.581865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1181/2359] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:45.583106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:45.584693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:45.616907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.617589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.618585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.639280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:45.640509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:45.646344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:45.662638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd0b43-3347-4bbc-ac74-3bb5bdb250a8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:45.668697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:45.669541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.669987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.670376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.671296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:45.674078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:45.703990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1182/2360] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:02:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:02:45.705839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1179/2361] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:02:45 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:02:45.709977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:45.716029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:45.718002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:45.722645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d9e1-2104-412e-9fb1-86f7c298f395 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:45.728864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:45.784563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:45.785171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.785451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.785716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.786372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:45.788312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:45.793516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde2313-5f65-4191-b00a-52280278a40d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:45.799182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:45.799906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.800191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.800470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.809959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1183/2362] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:02:45 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:02:45.814394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:45.825194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641488ff-d674-44d4-b7de-7c0a8919914f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:45.830319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:45.831326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.831608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.831882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.832901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:45.834524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:45.846800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1184/2363] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:02:45.850927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:45.862349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98831474-1115-4cc3-8edd-7d05e67e54e4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:45.864429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:45.866038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:45.867412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:45.868074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:45.868497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:45.868785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] PATH_INFO: `/v3/projects/c9264b7ff15649459f7599559d4fd273/users/f580c5ef6eed4edda4f31758d2cfab69/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:45.875778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1180/2364] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:02:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 15:02:45.885566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9264b7ff15649459f7599559d4fd273, user_id=f580c5ef6eed4edda4f31758d2cfab69, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:45.888560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:45.903157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1185/2365] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:02:45 2022] PUT /identity/v3/projects/c9264b7ff15649459f7599559d4fd273/users/f580c5ef6eed4edda4f31758d2cfab69/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:45.907134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:45.919761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f60f5-9163-427e-9dea-614ff606f60f service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:46.048847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e314f80-e401-4715-a12f-a91da7a28a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:46.050230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e314f80-e401-4715-a12f-a91da7a28a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:46.051408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e314f80-e401-4715-a12f-a91da7a28a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:46.051914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e314f80-e401-4715-a12f-a91da7a28a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:46.062004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e314f80-e401-4715-a12f-a91da7a28a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:46.062004 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e314f80-e401-4715-a12f-a91da7a28a77 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:46.078377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1181/2366] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:02:46.080533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:46.091993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711d48d-e30d-412a-ae00-49ab0e6f4690 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:46.098003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:46.098715 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:46.098957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:46.099351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] PATH_INFO: `/v3/projects/c9264b7ff15649459f7599559d4fd273/users/f580c5ef6eed4edda4f31758d2cfab69/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:46.101319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9264b7ff15649459f7599559d4fd273, user_id=f580c5ef6eed4edda4f31758d2cfab69, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:46.104082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:46.120821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1186/2367] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:02:46 2022] PUT /identity/v3/projects/c9264b7ff15649459f7599559d4fd273/users/f580c5ef6eed4edda4f31758d2cfab69/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:02:52.502765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f389c2-402a-4e85-b52a-d80aa68da89e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:52.503323 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f389c2-402a-4e85-b52a-d80aa68da89e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:52.508443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f389c2-402a-4e85-b52a-d80aa68da89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:52.564922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:52.568634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9f389c2-402a-4e85-b52a-d80aa68da89e None None] MFA Rules not processed for user `f580c5ef6eed4edda4f31758d2cfab69`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:52.570651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9f389c2-402a-4e85-b52a-d80aa68da89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:52.643498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1182/2368] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:52.657638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:02:52.659461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:52.806175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaabe4d-3826-42e7-8a15-5aba63a2c955 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:02:52.818796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2416f-6c85-48bd-a5af-b6605bb51e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:02:52.819766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2416f-6c85-48bd-a5af-b6605bb51e0c service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:52.820242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2416f-6c85-48bd-a5af-b6605bb51e0c service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:52.820695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2416f-6c85-48bd-a5af-b6605bb51e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:52.945580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:02:52.976327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2416f-6c85-48bd-a5af-b6605bb51e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:02:52.979917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2416f-6c85-48bd-a5af-b6605bb51e0c service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:02:53.003261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1187/2369] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:02:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:02:54.351023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0ffd0-9ec7-47cd-871f-17dcd7ef5ca2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:54.351555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0ffd0-9ec7-47cd-871f-17dcd7ef5ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:54.351555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0ffd0-9ec7-47cd-871f-17dcd7ef5ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:54.383299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e3b0ffd0-9ec7-47cd-871f-17dcd7ef5ca2 None None] MFA Rules not processed for user `c8a3f8bef9f0431eac090f8a520f744b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:54.384553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e3b0ffd0-9ec7-47cd-871f-17dcd7ef5ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:54.401005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1183/2370] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:02:55.672192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:02:55.672748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:02:55.672748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:02:55.742489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] MFA Rules not processed for user `482233dae68d487ca9b2ff6c69684663`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:02:55.744408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:02:55.860831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1188/2371] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:03:04.367630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=31 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:03:05.852238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=31 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:03:12.405121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:03:12.405639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:03:12.405742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:03:12.425316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:03:12.426707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:03:12.483117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1184/2372] 10.222.0.61 () {58 vars in 967 bytes} [Sat Sep 24 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:03:12.502915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:03:12.570295 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add9259-4fe0-4741-b57e-f019bf248889 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:03:12.575207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:03:12.575822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:03:12.576106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:03:12.576389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:03:12.584668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:03:12.586304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:03:12.594453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1189/2373] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:03:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:03:13.289695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:03:13.291126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:03:13.360129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a25053-c2df-4a8d-bb7f-8de129c9f90d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:03:13.367047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:03:13.367653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:03:13.367939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:03:13.368214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:03:13.370214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:03:13.382093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:03:13.383613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090bad3-af91-43d8-94ac-29a5a7b3ff71 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:03:13.393372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1185/2374] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:03:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:03:22.591197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=32 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:03:23.389169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=57 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:03:32.888569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:03:32.890049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:03:32.947904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8763e-3634-4f81-8ef0-bc2e36d41814 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:03:32.951587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11d89f-50ab-4ee2-8c81-edaa62afef26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:03:32.952133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d89f-50ab-4ee2-8c81-edaa62afef26 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:03:32.952414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d89f-50ab-4ee2-8c81-edaa62afef26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:03:32.952691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11d89f-50ab-4ee2-8c81-edaa62afef26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:03:32.962095 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11d89f-50ab-4ee2-8c81-edaa62afef26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:03:32.963527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11d89f-50ab-4ee2-8c81-edaa62afef26 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:03:32.973392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1190/2375] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:03:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:03:42.968650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=26 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:03:45.306312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:03:45.306835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:03:45.306943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:03:45.326933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] MFA Rules not processed for user `9f74f51060554db18bc96199222e9261`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:03:45.328736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:03:45.383925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1186/2376] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:03:55.377205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=25 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:03:59.195685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:03:59.196231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:03:59.196334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:03:59.216306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] MFA Rules not processed for user `db5a0bddb68047d68f0c2dd1af0cd57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:03:59.217507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:03:59.270993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1191/2377] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:05.966510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:05.969161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:06.052138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0390-b73a-41bc-99e4-392d344cb687 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:06.057219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:06.057878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.058195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.058570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] PATH_INFO: `/v3/users/482233dae68d487ca9b2ff6c69684663` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.059657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=482233dae68d487ca9b2ff6c69684663)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:06.061955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:06.131945 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482233dae68d487ca9b2ff6c69684663'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.141544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482233dae68d487ca9b2ff6c69684663'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.157098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482233dae68d487ca9b2ff6c69684663'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.157838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482233dae68d487ca9b2ff6c69684663'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.165583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f6785f8-743c-4b06-be9a-2a848d67eab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482233dae68d487ca9b2ff6c69684663'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.175531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1187/2378] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:04:05 2022] DELETE /identity/v3/users/482233dae68d487ca9b2ff6c69684663 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:04:06.485774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:06.501221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de34ad9-e20a-400f-806d-a667c30f0303 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:06.570289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:06.570964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.571339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.571550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] PATH_INFO: `/v3/projects/5e00eaec5c3a4b53b9e39346e792da01` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.572885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e00eaec5c3a4b53b9e39346e792da01)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:06.574911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:06.625137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e00eaec5c3a4b53b9e39346e792da01', 'request_id': 'req-7a123a0b-b11e-4a91-a5e3-ff90da369d56'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.630950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e00eaec5c3a4b53b9e39346e792da01', 'request_id': 'req-7a123a0b-b11e-4a91-a5e3-ff90da369d56'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.637557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e00eaec5c3a4b53b9e39346e792da01', 'request_id': 'req-7a123a0b-b11e-4a91-a5e3-ff90da369d56'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.650757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] The token cache is being invalidate because project 5e00eaec5c3a4b53b9e39346e792da01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:04:06.651050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.651709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7a123a0b-b11e-4a91-a5e3-ff90da369d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:04:06.653196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1192/2379] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:04:06 2022] DELETE /identity/v3/projects/5e00eaec5c3a4b53b9e39346e792da01 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:04:06.658994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.659413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.659764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.680435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:06.681648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:06.741710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1188/2380] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:06.751090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.751381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.751658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.771515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:06.772713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:06.780808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1193/2381] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:06.785082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:06.794599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482f18-7f9c-4509-8acb-9c26fa1e8814 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:06.799311 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:06.799911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.800192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.800463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.801111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:04:06.801449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:06.803160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:06.815568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1189/2382] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:04:06.819686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:06.829464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d3155-9636-4ca6-93e7-04cfd454ffd4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:06.834116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:06.834716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.835000 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.835277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.835937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:06.837849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:06.864509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1194/2383] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:04:06 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:04:06.868880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:06.879466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845d69f-29c6-48ab-a356-19bd6ee9996a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:06.882925 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:06.935568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:06.936200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.936501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.936772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.938056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:06.939962 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:06.960200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1190/2384] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:04:06 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:04:06.964356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:06.974541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cfaea-322b-46de-a831-b0629a97503a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:06.979167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:06.979756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:06.980040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:06.980315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:06.980956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:06.982629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:06.983854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:06.995041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1195/2385] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:04:06.998291 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:07.008619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52684c-264e-4c27-ba9d-8f855c2b8626 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:07.013106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397620e9-6233-406b-9238-5eaccfcc3390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:07.013683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397620e9-6233-406b-9238-5eaccfcc3390 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:07.014003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397620e9-6233-406b-9238-5eaccfcc3390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:07.014277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397620e9-6233-406b-9238-5eaccfcc3390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:07.014908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397620e9-6233-406b-9238-5eaccfcc3390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:07.016537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397620e9-6233-406b-9238-5eaccfcc3390 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:07.026842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1191/2386] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:04:07.030844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:07.041582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97da809-787f-49c0-b8dc-2f399ec36e46 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:07.047710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:07.048370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:07.048650 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:07.048947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] PATH_INFO: `/v3/projects/1afc2cc56fef4d5aa0e0905ef059d123/users/4e9a50200076443d98841a76e961408b/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:07.064149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1afc2cc56fef4d5aa0e0905ef059d123, user_id=4e9a50200076443d98841a76e961408b, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:07.066790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:07.080042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1196/2387] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:04:07 2022] PUT /identity/v3/projects/1afc2cc56fef4d5aa0e0905ef059d123/users/4e9a50200076443d98841a76e961408b/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:04:07.085384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0a7a1-7fac-4fb3-8b79-1decbab7397e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:07.085671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0a7a1-7fac-4fb3-8b79-1decbab7397e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:07.085984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0a7a1-7fac-4fb3-8b79-1decbab7397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:07.104455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-92b0a7a1-7fac-4fb3-8b79-1decbab7397e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:07.105636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-92b0a7a1-7fac-4fb3-8b79-1decbab7397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:07.163608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1192/2388] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:07.175059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:07.176771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:07.234997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88834abf-343c-49cb-9e21-41ba20d49a6d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:07.239183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:07.240152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:07.240451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:07.240708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:07.250320 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:07.251768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:07.261122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1197/2389] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:04:12.621758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:12.622286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:12.622382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:12.673618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] MFA Rules not processed for user `4e9a50200076443d98841a76e961408b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:12.674967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:12.797005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1193/2390] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:12.799814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:12.844518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63399d8c-d6f7-4942-9a4e-580248752013 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:12.879496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:12.880122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:12.880406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:12.881254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:12.881917 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:12.883858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:12.940802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1198/2391] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:04:12 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:04:12.945561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:12.976711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158844e-ab6f-4da5-bb8f-3033521d44ec None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:13.000061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:13.058585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:13.059416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:13.059873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:13.060074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:13.061101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:13.063273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:13.096389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1194/2392] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:04:12 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:04:13.100626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:13.116185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709b8f0-55b3-443b-b64f-1ed33e817488 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:13.123499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:13.124458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:13.125708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:13.126074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:13.126866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:13.128604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:13.149071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1199/2393] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:04:13.153171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:13.165125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6ecf7-9592-4852-be3b-a239b69fd928 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:13.175335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:13.176090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:13.176443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:13.176727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] PATH_INFO: `/v3/projects/555986b504ca4282bcee2de77528f1ba/users/2ff5db15af92475aa6219d9e28914652/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:13.195626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] RBAC: Authorizing `identity:create_grant(project_id=555986b504ca4282bcee2de77528f1ba, user_id=2ff5db15af92475aa6219d9e28914652, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:13.198572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:13.218625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1195/2394] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:04:13 2022] PUT /identity/v3/projects/555986b504ca4282bcee2de77528f1ba/users/2ff5db15af92475aa6219d9e28914652/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:04:13.219636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:13.241043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6eb22-9102-4821-b7c1-74f8d6a44c3f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:13.310509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0674-0364-4768-861e-c64cb5de61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:13.311170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0674-0364-4768-861e-c64cb5de61d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:13.311464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0674-0364-4768-861e-c64cb5de61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:13.312957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0674-0364-4768-861e-c64cb5de61d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:13.313681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0674-0364-4768-861e-c64cb5de61d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:13.315484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0674-0364-4768-861e-c64cb5de61d0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:13.326491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1200/2395] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:04:13.330944 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:13.342306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaed55-505c-42de-815d-f5490484935f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:13.348711 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffc7aa-dc5c-4124-9be5-6b95d2522033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:13.349430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffc7aa-dc5c-4124-9be5-6b95d2522033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:13.349715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffc7aa-dc5c-4124-9be5-6b95d2522033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:13.350080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffc7aa-dc5c-4124-9be5-6b95d2522033 admin admin] PATH_INFO: `/v3/projects/555986b504ca4282bcee2de77528f1ba/users/2ff5db15af92475aa6219d9e28914652/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:13.352065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffc7aa-dc5c-4124-9be5-6b95d2522033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555986b504ca4282bcee2de77528f1ba, user_id=2ff5db15af92475aa6219d9e28914652, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:13.354900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffc7aa-dc5c-4124-9be5-6b95d2522033 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:13.367749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1196/2396] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:04:13 2022] PUT /identity/v3/projects/555986b504ca4282bcee2de77528f1ba/users/2ff5db15af92475aa6219d9e28914652/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:04:17.504518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:17.505586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:17.505893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:17.535446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] MFA Rules not processed for user `cdbc898ffd4f4d5d87c3684fcc17313d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:17.537299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:17.590921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:17.633189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1201/2397] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:18.669384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda09b67-0d27-4c91-93d4-b299fce2fc89 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:18.669781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda09b67-0d27-4c91-93d4-b299fce2fc89 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:18.670073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda09b67-0d27-4c91-93d4-b299fce2fc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:18.700497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eda09b67-0d27-4c91-93d4-b299fce2fc89 None None] MFA Rules not processed for user `2ff5db15af92475aa6219d9e28914652`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:18.701740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eda09b67-0d27-4c91-93d4-b299fce2fc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:18.777547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1197/2398] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:18.790318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:18.792706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:18.883186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a2ee-9f99-4d6b-b2df-f53eb28b325d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:18.889059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:18.889833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:18.890937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:18.891310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:18.985944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:18.987049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:18.996667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1202/2399] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:04:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:04:19.159251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ef52f-09ab-4326-b28e-1da9235400d9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:19.159538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ef52f-09ab-4326-b28e-1da9235400d9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:19.159817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ef52f-09ab-4326-b28e-1da9235400d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:19.180136 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-af3ef52f-09ab-4326-b28e-1da9235400d9 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:19.181580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af3ef52f-09ab-4326-b28e-1da9235400d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:19.237394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:19.252558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1198/2400] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:19.308064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:19.309923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:19.409077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caec3c-f984-4d30-9c89-5b510dededca service glance] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:19.417086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e75a7-6830-44b3-af1f-e270bb71b6f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:19.417804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e75a7-6830-44b3-af1f-e270bb71b6f8 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:19.418201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e75a7-6830-44b3-af1f-e270bb71b6f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:19.418547 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e75a7-6830-44b3-af1f-e270bb71b6f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:19.433503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e75a7-6830-44b3-af1f-e270bb71b6f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:19.436205 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e75a7-6830-44b3-af1f-e270bb71b6f8 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:19.449190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1203/2401] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:04:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:04:29.252620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:29.276568 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:29.276874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:29.277149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:29.298397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] MFA Rules not processed for user `4e9a50200076443d98841a76e961408b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:29.299749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:29.306617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1199/2402] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:04:29.445290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:32.162235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:32.162905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:32.163159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:32.198040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:04:32.199508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:32.207565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1204/2403] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:04:32.220689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:32.234633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebef47-177c-4b40-a0fb-4884f4340362 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:32.239871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:32.241436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:32.241436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:32.241436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:32.252051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:32.253642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:32.262474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1200/2404] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:04:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:04:42.185116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:42.258858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=9 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:04:53.326137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:53.384200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161213-e131-4c19-bab1-ff362daa3f69 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:53.388495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:53.389210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:53.389556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:53.389890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:53.392325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:53.477160 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:53.478735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:53.490724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1205/2405] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:04:55.312075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:04:55.314076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:55.372556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3f6dd-9e49-4154-9649-a72852a6b664 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:04:55.376855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:04:55.377487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:04:55.377791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:04:55.378094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:04:55.379825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:04:55.394345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:04:55.395771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:04:55.406687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1201/2406] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:04:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:00.286053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:00.304693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacdffa-2356-4854-94f1-4da8cac88e66 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:00.308740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:00.309351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:00.309631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:00.309925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:00.311816 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:00.384355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:00.386094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:00.398904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1206/2407] 10.222.0.61 () {60 vars in 1396 bytes} [Sat Sep 24 15:05:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:05.402261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=26 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:07.011533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:07.012957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:07.032695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa87d-2ec6-45b7-8a69-0669e5ea37e1 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:07.039285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:07.039888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:07.040174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:07.040448 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:07.042203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:07.107118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:07.108708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:07.118502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1202/2408] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:07.700870 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:07.710600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad11ce-49a6-47e1-8bcd-8c02e5afa41b service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:07.714478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:07.715077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:07.715460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:07.715736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:07.725478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:07.726948 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:07.736237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1207/2409] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:08.285243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:08.286693 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:08.299243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f5a2d-50c2-4217-84ff-a37a899b7a3c service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:08.302866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:08.303466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:08.303751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:08.304026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:08.305730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:08.381268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:08.382727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:08.394254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1203/2410] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:12.980689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:12.982393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:13.003422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50cb55-77ca-44f4-99d5-b13daee46e80 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.009321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.010791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.011252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.011708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] PATH_INFO: `/v3/users/db5a0bddb68047d68f0c2dd1af0cd57b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.013155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5a0bddb68047d68f0c2dd1af0cd57b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.016308 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.060841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:13.115090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5a0bddb68047d68f0c2dd1af0cd57b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.132136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5a0bddb68047d68f0c2dd1af0cd57b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.148801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5a0bddb68047d68f0c2dd1af0cd57b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.149998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5a0bddb68047d68f0c2dd1af0cd57b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.162241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3546f291-bf6e-4b30-8ca6-e48f77fea201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5a0bddb68047d68f0c2dd1af0cd57b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.173606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1208/2411] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:12 2022] DELETE /identity/v3/users/db5a0bddb68047d68f0c2dd1af0cd57b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:13.383593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:13.401277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916e9b5-b280-4283-81b2-737242e0e6ef service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.482383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.483038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.483522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.484000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] PATH_INFO: `/v3/projects/578268f34d874bf9835e8ec4cb31ebee` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.485460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=578268f34d874bf9835e8ec4cb31ebee)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.487665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.533618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:13.544844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578268f34d874bf9835e8ec4cb31ebee', 'request_id': 'req-0b208c7c-4099-4613-a90b-eba6b69d2d7f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.552465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578268f34d874bf9835e8ec4cb31ebee', 'request_id': 'req-0b208c7c-4099-4613-a90b-eba6b69d2d7f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.559845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578268f34d874bf9835e8ec4cb31ebee', 'request_id': 'req-0b208c7c-4099-4613-a90b-eba6b69d2d7f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.585869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] The token cache is being invalidate because project 578268f34d874bf9835e8ec4cb31ebee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:13.586235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.586876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0b208c7c-4099-4613-a90b-eba6b69d2d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:13.588388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1204/2412] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:13 2022] DELETE /identity/v3/projects/578268f34d874bf9835e8ec4cb31ebee => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:13.593790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.594109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.594385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.619100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:13.620304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:13.693359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1209/2413] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:13.701320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.701608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.701888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.722591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:13.723805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:13.732388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1205/2414] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:13.736801 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:13.748378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8efc27-9a28-422f-8277-835fd63de0c6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.753204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.753936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.754286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.754645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.755413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:13.755782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.757484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.770092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1210/2415] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:13.774587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:13.785325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2abad-e2e9-4feb-b5a3-15cf9a33a50a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.790191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.790878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.791227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.791718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.792433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.794440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.822618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1206/2416] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:13 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:13.826852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:13.838380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f4ed3-3653-4666-9c59-f16059e1a4ce admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.900373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.900926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.901215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.901493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.904421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.905147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.930844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1211/2417] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:13 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:13.935023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:13.945297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6d88a-8fcf-4d47-a3d5-cf330f2f8869 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.949982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.950610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.950891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.951165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.951813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.953960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.964334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1207/2418] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:13.969157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:13.979865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a70daa-5a59-474d-b9e7-9b349fba379f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:13.984559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d6ce6-ed11-4456-a0ad-cdbe256fc4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:13.985161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6ce6-ed11-4456-a0ad-cdbe256fc4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:13.985417 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6ce6-ed11-4456-a0ad-cdbe256fc4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:13.985828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6ce6-ed11-4456-a0ad-cdbe256fc4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:13.986585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d6ce6-ed11-4456-a0ad-cdbe256fc4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:13.988193 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d6ce6-ed11-4456-a0ad-cdbe256fc4c3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:13.999049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1212/2419] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:14.003413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.013652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6c39-d5ad-4cdf-a889-a52f272924f2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.019083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.019740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.020028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.020309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] PATH_INFO: `/v3/projects/b341e8eef14c4e54bf9d4ae93337f23a/users/4b34e27e463048439557c5b191cb44ae/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.036178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b341e8eef14c4e54bf9d4ae93337f23a, user_id=4b34e27e463048439557c5b191cb44ae, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.038871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.053158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1208/2420] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:14 2022] PUT /identity/v3/projects/b341e8eef14c4e54bf9d4ae93337f23a/users/4b34e27e463048439557c5b191cb44ae/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:14.065010 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.065318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.065601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.073891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:14.097098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] MFA Rules not processed for user `4b34e27e463048439557c5b191cb44ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:14.098508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:14.167703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1213/2421] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:14.172341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.238298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04ea8-416c-4579-8fd6-42cc93810fc7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.243363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.243972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.244256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.244530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.245198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.247131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.278110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1209/2422] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:14 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:14.279267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.289887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4ad05-1bc6-4dc4-9d94-b61589486232 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.363737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.364361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.364665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.365953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.366609 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.369167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.396019 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1214/2423] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:14 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:05:14.397221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.407811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed841cd-05d6-4abf-8ab4-f161f9c323ce admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.412537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.413154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.413439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.413715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.414482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.416271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.428278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1210/2424] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:14.432623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.443868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77730057-f63f-438b-8cec-5ac14af51c29 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.448665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.449343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.449627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.449922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] PATH_INFO: `/v3/projects/4509a1f6e65b4af8a00d05b59fdc255a/users/2b4b30b20c254a9694b3cc157d98e1c3/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.466423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4509a1f6e65b4af8a00d05b59fdc255a, user_id=2b4b30b20c254a9694b3cc157d98e1c3, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.469220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.484580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1215/2425] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:14 2022] PUT /identity/v3/projects/4509a1f6e65b4af8a00d05b59fdc255a/users/2b4b30b20c254a9694b3cc157d98e1c3/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:14.488624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.500322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5298edd-8f90-4a7e-a8f8-43d5e4215410 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.532649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:14.563360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198dc15-b6f1-4d67-a8c6-e510ced63464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.563977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198dc15-b6f1-4d67-a8c6-e510ced63464 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.564259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198dc15-b6f1-4d67-a8c6-e510ced63464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.564565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198dc15-b6f1-4d67-a8c6-e510ced63464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.565212 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198dc15-b6f1-4d67-a8c6-e510ced63464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.566857 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198dc15-b6f1-4d67-a8c6-e510ced63464 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.577493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1211/2426] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:14.582405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.593279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8ae62-4a8b-46ff-a53d-8bf7bda92312 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.598247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.599079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.599362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.599635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] PATH_INFO: `/v3/projects/4509a1f6e65b4af8a00d05b59fdc255a/users/2b4b30b20c254a9694b3cc157d98e1c3/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.601553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4509a1f6e65b4af8a00d05b59fdc255a, user_id=2b4b30b20c254a9694b3cc157d98e1c3, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.604291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.616319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1216/2427] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:14 2022] PUT /identity/v3/projects/4509a1f6e65b4af8a00d05b59fdc255a/users/2b4b30b20c254a9694b3cc157d98e1c3/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:14.627159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.627525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.627873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.657864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] MFA Rules not processed for user `2b4b30b20c254a9694b3cc157d98e1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:14.659161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:14.739357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1212/2428] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:14.752607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:14.754332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:14.807484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:14.814871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70350ed9-8e2d-4550-b8dc-71dfa1bc9f6c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:14.818605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880883a-259f-4dbc-adf9-e11c3c329c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:14.819210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880883a-259f-4dbc-adf9-e11c3c329c67 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:14.819490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880883a-259f-4dbc-adf9-e11c3c329c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:14.819762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880883a-259f-4dbc-adf9-e11c3c329c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:14.829339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880883a-259f-4dbc-adf9-e11c3c329c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:14.830864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880883a-259f-4dbc-adf9-e11c3c329c67 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:14.840818 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1217/2429] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:15.284342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:15.350112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a781-f8bc-4a9d-a2e4-1df096cb91fa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:15.354813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:15.355444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:15.355727 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:15.356005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] PATH_INFO: `/v3/users/4b34e27e463048439557c5b191cb44ae` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:15.357123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b34e27e463048439557c5b191cb44ae)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:15.359610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:15.421740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b34e27e463048439557c5b191cb44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:15.433009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b34e27e463048439557c5b191cb44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:15.436455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:15.450963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b34e27e463048439557c5b191cb44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:15.451899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b34e27e463048439557c5b191cb44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:15.459704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b34e27e463048439557c5b191cb44ae'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:15.465881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1213/2430] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:15 2022] DELETE /identity/v3/users/4b34e27e463048439557c5b191cb44ae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:15.471196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:15.471709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:15.471709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:15.494617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:15.496143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:15.563652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1218/2431] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:15.573131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:15.574641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:15.632955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bee6-a7f1-4ee2-8bc3-1383c06e50be admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:15.637795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fefcff-86af-465a-988d-e0538a377bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:15.638776 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fefcff-86af-465a-988d-e0538a377bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:15.639065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fefcff-86af-465a-988d-e0538a377bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:15.639330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fefcff-86af-465a-988d-e0538a377bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:15.650896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fefcff-86af-465a-988d-e0538a377bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:15.652277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fefcff-86af-465a-988d-e0538a377bc0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:15.662991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1214/2432] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:16.068408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:16.070090 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.082764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38735631-fa35-40bb-b98b-8fca555c7dd4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:16.087481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:16.088114 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.088511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.088792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] PATH_INFO: `/v3/projects/b341e8eef14c4e54bf9d4ae93337f23a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.089940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b341e8eef14c4e54bf9d4ae93337f23a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:16.091871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:16.130750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.131040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.131320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.154988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b341e8eef14c4e54bf9d4ae93337f23a', 'request_id': 'req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.155277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] MFA Rules not processed for user `9f74f51060554db18bc96199222e9261`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:16.157106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.162348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b341e8eef14c4e54bf9d4ae93337f23a', 'request_id': 'req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.169696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b341e8eef14c4e54bf9d4ae93337f23a', 'request_id': 'req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.184061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] The token cache is being invalidate because project b341e8eef14c4e54bf9d4ae93337f23a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:16.184338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.184984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.187281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1219/2433] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:16 2022] DELETE /identity/v3/projects/b341e8eef14c4e54bf9d4ae93337f23a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:16.191516 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:16.192937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.225790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1215/2434] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:16.268692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1ac8e-08e7-4b3b-8ea2-900ef618ef40 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:16.275062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:16.275690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.275975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.276252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] PATH_INFO: `/v3/users/2b4b30b20c254a9694b3cc157d98e1c3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.277377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4b30b20c254a9694b3cc157d98e1c3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:16.279331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:16.296042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:16.331785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4b30b20c254a9694b3cc157d98e1c3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.339990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4b30b20c254a9694b3cc157d98e1c3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.352512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4b30b20c254a9694b3cc157d98e1c3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.353252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4b30b20c254a9694b3cc157d98e1c3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.360137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4b30b20c254a9694b3cc157d98e1c3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.366336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1220/2435] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:16 2022] DELETE /identity/v3/users/2b4b30b20c254a9694b3cc157d98e1c3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:16.550951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:16.552487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.611518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a5bef-e04d-42a8-9d1e-84151875e7ee None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:16.642331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:16.643915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.654317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:16.674643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:16.675254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.675538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.675813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.677662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.687209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:16.688632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36363e5-bc66-4cba-bd15-9437faf5873a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:16.696863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1216/2436] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:05:16.715038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64357300-ac54-4737-ac8a-c74ab44042dc admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:16.720371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:16.721001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.721283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.721560 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] PATH_INFO: `/v3/projects/4509a1f6e65b4af8a00d05b59fdc255a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.722704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4509a1f6e65b4af8a00d05b59fdc255a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:16.724630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:16.774288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4509a1f6e65b4af8a00d05b59fdc255a', 'request_id': 'req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.780225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4509a1f6e65b4af8a00d05b59fdc255a', 'request_id': 'req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.787500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4509a1f6e65b4af8a00d05b59fdc255a', 'request_id': 'req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.800552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] The token cache is being invalidate because project 4509a1f6e65b4af8a00d05b59fdc255a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:16.800845 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.801473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-edcb65f4-1b0e-4636-9c6d-fc34aa689eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:16.803028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1221/2437] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:16 2022] DELETE /identity/v3/projects/4509a1f6e65b4af8a00d05b59fdc255a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:05:16.808069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.808567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.808849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.828618 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:16.829806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.891317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1217/2438] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:05:16.899720 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.900128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.900465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.920639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:16.922149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:16.933613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1222/2439] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:16.934851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:16.944764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368246a3-688e-43e5-a75c-b5f300b7cd32 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:16.949616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:16.950325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.950598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.950875 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.951592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:16.951924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:16.954098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:16.966597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1218/2440] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:16.971352 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:16.980763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9a023-e13a-4f16-946c-46c098ea6d74 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:16.986251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:16.986862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:16.987149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:16.987423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:16.988060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:16.989985 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.007898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:17.022568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1223/2441] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:16 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:17.026553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.035441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.037327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:17.041710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501c7b8-094f-4db0-b712-1eb9b5a5faf9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.110763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539f97-1d6d-43d5-afaa-526eb169a681 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.112352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.113063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.113419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.113766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.114504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.116358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.116686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.116964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.117276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.117552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.119333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:17.132940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.134797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.139818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1219/2442] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:17 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:17.144352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.148476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1224/2443] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:17.154097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392d40-2eeb-4c8a-bbf9-eb778dbd4576 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.158956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.159548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.159836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.160110 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.160746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.162362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.173351 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1220/2444] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:05:17.177456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.188199 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ce4b3-0724-4008-8971-1bf0daa1c06c service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.192795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706a1b-134f-44fc-bbe4-caa47ecc6429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.193390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706a1b-134f-44fc-bbe4-caa47ecc6429 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.193674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706a1b-134f-44fc-bbe4-caa47ecc6429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.194217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706a1b-134f-44fc-bbe4-caa47ecc6429 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.194726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe706a1b-134f-44fc-bbe4-caa47ecc6429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.196324 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe706a1b-134f-44fc-bbe4-caa47ecc6429 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.206984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1225/2445] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:17.212090 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.224301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f26dc1-9bff-422e-9bf2-cd0e453e1684 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.229617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.230325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.231412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.231689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] PATH_INFO: `/v3/projects/a76c6797314e4afe874350b6275bcb07/users/079d3345b9b64c858446271893c050d9/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.247096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a76c6797314e4afe874350b6275bcb07, user_id=079d3345b9b64c858446271893c050d9, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.250325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.265998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1221/2446] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:17 2022] PUT /identity/v3/projects/a76c6797314e4afe874350b6275bcb07/users/079d3345b9b64c858446271893c050d9/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:17.275721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.275834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.276094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.304284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] MFA Rules not processed for user `079d3345b9b64c858446271893c050d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:17.305494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:17.362096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.363736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:17.375513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1226/2447] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:17.379768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.400104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:17.454184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80de7b-601a-4e6f-bd31-c913aa256795 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.459616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.460287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.460506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.460781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.461416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.463356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.464622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54df112-4b9d-40c4-bb60-0e74cc81a738 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.468840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:17.474521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.475753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.476850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.477522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.480447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:17.497947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1227/2448] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:17 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:05:17.500781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.549228 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.550806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.559956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1222/2449] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:17.575044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d848ce0-f065-4448-9f5a-4b8f8a5d4eb7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.580871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.581495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.581776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.582101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.582764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.584685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.609436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1228/2450] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:17 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:05:17.613091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.629865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241f6dc-5f09-48bc-951c-d327da92fad8 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.634825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.635686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.636118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.636480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.637262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.638975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.651721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1223/2451] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:17.656166 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.666167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9c02e-1db9-4a53-a3d1-9ddca2ebff8e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.670928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.672106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.672386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.672820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] PATH_INFO: `/v3/projects/6720a09bdb5442a3b085ba9a19eba42d/users/e0c598272d884d76b31af0045e7f8d40/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.689291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6720a09bdb5442a3b085ba9a19eba42d, user_id=e0c598272d884d76b31af0045e7f8d40, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.691940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.705646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1229/2452] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:17 2022] PUT /identity/v3/projects/6720a09bdb5442a3b085ba9a19eba42d/users/e0c598272d884d76b31af0045e7f8d40/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:17.711027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.721391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950f606-c8bc-4926-8087-4aa5f1118f09 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.789979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf35428-548f-4ca4-bf1d-ce1875101774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.790609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf35428-548f-4ca4-bf1d-ce1875101774 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.790898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf35428-548f-4ca4-bf1d-ce1875101774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.791177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf35428-548f-4ca4-bf1d-ce1875101774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.792132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf35428-548f-4ca4-bf1d-ce1875101774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.793793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf35428-548f-4ca4-bf1d-ce1875101774 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.808007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1224/2453] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:17.816311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:17.828822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16679f0-a4ba-4bf7-b4cc-fadf23c9da2b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:17.834372 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:17.835785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.836312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.836579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] PATH_INFO: `/v3/projects/6720a09bdb5442a3b085ba9a19eba42d/users/e0c598272d884d76b31af0045e7f8d40/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.838761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6720a09bdb5442a3b085ba9a19eba42d, user_id=e0c598272d884d76b31af0045e7f8d40, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:17.841407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:17.853752 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1230/2454] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:17 2022] PUT /identity/v3/projects/6720a09bdb5442a3b085ba9a19eba42d/users/e0c598272d884d76b31af0045e7f8d40/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:05:17.864327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:17.865012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:17.865012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:17.911601 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] MFA Rules not processed for user `e0c598272d884d76b31af0045e7f8d40`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:17.912964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:17.967799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:18.008362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1225/2455] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:18.019847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:18.021686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:18.079924 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee278-0c80-4ba6-8e9e-82e89d092b68 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:18.083477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:18.084076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:18.084361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:18.084635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:18.094385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:18.096448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:18.106542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1231/2456] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:18.153382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:18.154804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:18.167516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34dcaa-6bbc-4009-9d6a-758f4f1b0fce None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:18.171723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:18.172305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:18.172589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:18.172863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:18.228676 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:18.230132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:18.239384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1226/2457] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:19.438982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:19.441581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:19.504683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:19.523158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968faec-4e59-4ff8-acf9-d5ba897e54db service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:19.527440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:19.528066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:19.528354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:19.528624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:19.530988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:19.541168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:19.542749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de95a8d-351e-46cf-a6b2-4f2fb77af385 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:19.552300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1232/2458] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:21.630642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:21.694360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077392d-497e-4732-b4a4-965b685e7749 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:21.699153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:21.699812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:21.700092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:21.700369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] PATH_INFO: `/v3/users/079d3345b9b64c858446271893c050d9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:21.701550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=079d3345b9b64c858446271893c050d9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:21.704184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:21.756753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079d3345b9b64c858446271893c050d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:21.767416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079d3345b9b64c858446271893c050d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:21.778617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:21.780485 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079d3345b9b64c858446271893c050d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:21.781174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079d3345b9b64c858446271893c050d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:21.790129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079d3345b9b64c858446271893c050d9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:21.796973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1227/2459] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:21 2022] DELETE /identity/v3/users/079d3345b9b64c858446271893c050d9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:21.801333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:21.801766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:21.801988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:21.826931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:21.828699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:21.891171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1233/2460] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:21.901041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:21.902598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:21.966723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad1c4d-e8fe-4907-bef4-61e5b44ee5f4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:21.972932 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:21.973537 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:21.973815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:21.974107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:21.983561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:21.984984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:21.994536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1228/2461] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:22.289156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:22.290618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:22.302537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea0b6-bab3-4709-9bbc-40aeb5d2cc42 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:22.308127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:22.308775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:22.309085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:22.309398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] PATH_INFO: `/v3/projects/a76c6797314e4afe874350b6275bcb07` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:22.311080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a76c6797314e4afe874350b6275bcb07)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:22.313096 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:22.374025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a76c6797314e4afe874350b6275bcb07', 'request_id': 'req-5a09d186-eb68-4b80-b53e-7eb445fc0078'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.380244 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a76c6797314e4afe874350b6275bcb07', 'request_id': 'req-5a09d186-eb68-4b80-b53e-7eb445fc0078'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.386589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a76c6797314e4afe874350b6275bcb07', 'request_id': 'req-5a09d186-eb68-4b80-b53e-7eb445fc0078'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.399572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] The token cache is being invalidate because project a76c6797314e4afe874350b6275bcb07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:22.399861 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.400484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.401977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1234/2462] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:22 2022] DELETE /identity/v3/projects/a76c6797314e4afe874350b6275bcb07 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:22.405967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:22.421184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6642c-137b-4976-a606-52cd166ab242 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:22.476784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:22.477402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:22.477680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:22.477981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] PATH_INFO: `/v3/users/e0c598272d884d76b31af0045e7f8d40` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:22.479081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c598272d884d76b31af0045e7f8d40)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:22.480986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:22.524979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c598272d884d76b31af0045e7f8d40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.531804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c598272d884d76b31af0045e7f8d40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.544040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c598272d884d76b31af0045e7f8d40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.544644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c598272d884d76b31af0045e7f8d40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.551536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-08a5871b-792e-4319-8227-4b550e9a718a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c598272d884d76b31af0045e7f8d40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.557338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1229/2463] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:22 2022] DELETE /identity/v3/users/e0c598272d884d76b31af0045e7f8d40 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:22.817447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:22.818980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:22.833799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d186-eb68-4b80-b53e-7eb445fc0078 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:22.894226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:22.894848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:22.895154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:22.895420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] PATH_INFO: `/v3/projects/6720a09bdb5442a3b085ba9a19eba42d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:22.896506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6720a09bdb5442a3b085ba9a19eba42d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:22.898437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:22.908450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:22.946290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6720a09bdb5442a3b085ba9a19eba42d', 'request_id': 'req-885e8be4-38f4-43ea-9926-e5bb92d53679'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.951758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6720a09bdb5442a3b085ba9a19eba42d', 'request_id': 'req-885e8be4-38f4-43ea-9926-e5bb92d53679'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.957892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6720a09bdb5442a3b085ba9a19eba42d', 'request_id': 'req-885e8be4-38f4-43ea-9926-e5bb92d53679'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.971753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] The token cache is being invalidate because project 6720a09bdb5442a3b085ba9a19eba42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:22.972154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.972888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-885e8be4-38f4-43ea-9926-e5bb92d53679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:22.974452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1235/2464] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:22 2022] DELETE /identity/v3/projects/6720a09bdb5442a3b085ba9a19eba42d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:22.979406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:22.979914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:22.980012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.001362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:23.002609 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:23.006909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:23.069193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1230/2465] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:23.076987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.077305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.078050 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.099978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:23.101234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:23.110180 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1236/2466] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:05:23.114659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.124706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2d48d-8b77-4873-aa8c-a77e23658551 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.129468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.130116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.130404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.130682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.131484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:23.131803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.133419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.145121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1231/2467] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:23.149521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.159451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea77168-87d1-4180-a183-e020dce46fee None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.164172 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.164802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.165085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.165361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.166027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.167936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.201577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1237/2468] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:23.202822 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.212746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909bb8-f1a8-4a43-a01c-9e254481e981 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.274238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.274979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.275358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.275694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.276383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.278392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.300853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1232/2469] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:23.305282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.316848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2d32b-ce34-4a83-b6b3-e05fd966697b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.321539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.322122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.322405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.322679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.323323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.324935 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.334986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1238/2470] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:23.338998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.349602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c9f64-4597-4466-9a12-90f2113cac02 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.354779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be80e59-e204-4349-ad63-2509816e11ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.355495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80e59-e204-4349-ad63-2509816e11ed admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.355788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80e59-e204-4349-ad63-2509816e11ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.356069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80e59-e204-4349-ad63-2509816e11ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.356723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be80e59-e204-4349-ad63-2509816e11ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.358368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be80e59-e204-4349-ad63-2509816e11ed admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.371722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1233/2471] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:05:23.376013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.386124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c20d30-53b8-4fb1-8c3d-1fabff239ccf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.390759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.391395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.391671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.391939 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] PATH_INFO: `/v3/projects/05230b9d2c1748ad950bfb39bc507bd3/users/5d1f9f4a1b284004a070194270debd5d/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.407245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05230b9d2c1748ad950bfb39bc507bd3, user_id=5d1f9f4a1b284004a070194270debd5d, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.409931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.424727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1239/2472] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:23 2022] PUT /identity/v3/projects/05230b9d2c1748ad950bfb39bc507bd3/users/5d1f9f4a1b284004a070194270debd5d/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:05:23.436435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.436697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.436975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.466200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] MFA Rules not processed for user `5d1f9f4a1b284004a070194270debd5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:23.467432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:23.535294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1234/2473] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:23.540256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.604233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc23a-f590-4e6c-bdd3-5cd71680f946 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.608811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.609415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.609697 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.610001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.610627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.612532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.637984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1240/2474] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:23.642731 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.649340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:23.653741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679abd2e-02a8-401c-9956-932c00de9c1b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.711312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.711924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.712236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.712512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.713164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.715144 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.737955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1235/2475] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:23.742623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.753518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d92e4e-2879-4619-888a-c0400682477b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.758117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.758710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.758993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.759256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.759892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.761508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.774280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1241/2476] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:23.782574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.791201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a82ff8-6064-4c9c-bb1c-746b4587a7c2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.796117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.796775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.797064 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.797346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] PATH_INFO: `/v3/projects/fb402ba741084c4f855f9ccc08b10229/users/3829ae7fdaca46e4b11024008525f47d/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.814306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb402ba741084c4f855f9ccc08b10229, user_id=3829ae7fdaca46e4b11024008525f47d, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.816993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.830761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1236/2477] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:23 2022] PUT /identity/v3/projects/fb402ba741084c4f855f9ccc08b10229/users/3829ae7fdaca46e4b11024008525f47d/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:23.834703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.844929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c3c8c-952e-4136-be8d-ba236922c9df admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.849510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:23.901629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9a12d-ce22-4ea2-98ab-747562beb735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.902181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a12d-ce22-4ea2-98ab-747562beb735 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.902459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a12d-ce22-4ea2-98ab-747562beb735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.902729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9a12d-ce22-4ea2-98ab-747562beb735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.903356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9a12d-ce22-4ea2-98ab-747562beb735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.904968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9a12d-ce22-4ea2-98ab-747562beb735 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.914443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1242/2478] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:23.918774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:23.929132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7625fe5-2f3a-4f2c-8e89-f43ebddad95d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:23.933821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:23.934642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.934929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.935205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] PATH_INFO: `/v3/projects/fb402ba741084c4f855f9ccc08b10229/users/3829ae7fdaca46e4b11024008525f47d/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.937634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb402ba741084c4f855f9ccc08b10229, user_id=3829ae7fdaca46e4b11024008525f47d, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:23.940301 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:23.950626 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1237/2479] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:23 2022] PUT /identity/v3/projects/fb402ba741084c4f855f9ccc08b10229/users/3829ae7fdaca46e4b11024008525f47d/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:23.961704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:23.962028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:23.962302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:23.991927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] MFA Rules not processed for user `3829ae7fdaca46e4b11024008525f47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:23.993137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:24.058042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1243/2480] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:24.069040 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:24.070786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:24.126576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4ff23-d285-4697-be18-89cbe613e486 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:24.130501 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:24.131092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:24.131368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:24.131637 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:24.141579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:24.143081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:24.152383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1238/2481] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:24.259665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:24.269666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42655-d561-4981-8583-d482a49f2a35 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:24.273613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2b9ec-d301-4921-a411-bbe65980b930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:24.274266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b9ec-d301-4921-a411-bbe65980b930 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:24.274565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b9ec-d301-4921-a411-bbe65980b930 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:24.274840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b9ec-d301-4921-a411-bbe65980b930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:24.331724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2b9ec-d301-4921-a411-bbe65980b930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:24.333165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2b9ec-d301-4921-a411-bbe65980b930 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:24.342130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1244/2482] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:27.076823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:27.139208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a9e36-1359-4833-9044-067fd65abc83 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:27.143726 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:27.144348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:27.144631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:27.144909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] PATH_INFO: `/v3/users/5d1f9f4a1b284004a070194270debd5d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:27.146045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1f9f4a1b284004a070194270debd5d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:27.147942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:27.149363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:27.192570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1f9f4a1b284004a070194270debd5d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.200000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1f9f4a1b284004a070194270debd5d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.212369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1f9f4a1b284004a070194270debd5d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.213041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1f9f4a1b284004a070194270debd5d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.220520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1f9f4a1b284004a070194270debd5d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.226310 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1239/2483] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:27 2022] DELETE /identity/v3/users/5d1f9f4a1b284004a070194270debd5d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:27.230782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:27.231285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:27.231394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:27.250313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:27.251518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:27.276060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:27.314290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1245/2484] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:27.323750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:27.325134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:27.381748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002d095-0177-4c18-b72f-fbb6a48a87a7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:27.385319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:27.385929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:27.386219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:27.386492 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:27.395751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:27.397160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:27.407022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1240/2485] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:27.667854 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:27.670134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:27.684112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12015cb-a784-4143-93fb-01f9dcc2eaa2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:27.689564 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:27.690376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:27.690660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:27.690934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] PATH_INFO: `/v3/projects/05230b9d2c1748ad950bfb39bc507bd3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:27.692360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05230b9d2c1748ad950bfb39bc507bd3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:27.694321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:27.744814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05230b9d2c1748ad950bfb39bc507bd3', 'request_id': 'req-16aa33f9-1bd6-43be-af1c-76607292a20a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.751797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05230b9d2c1748ad950bfb39bc507bd3', 'request_id': 'req-16aa33f9-1bd6-43be-af1c-76607292a20a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.758257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05230b9d2c1748ad950bfb39bc507bd3', 'request_id': 'req-16aa33f9-1bd6-43be-af1c-76607292a20a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.773546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] The token cache is being invalidate because project 05230b9d2c1748ad950bfb39bc507bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:27.773840 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.774510 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.776167 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1246/2486] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:27 2022] DELETE /identity/v3/projects/05230b9d2c1748ad950bfb39bc507bd3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:27.780160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:27.790794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8e61-8297-43df-bda3-ae2fdbd6c59c service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:27.853860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:27.854524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:27.854810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:27.855086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] PATH_INFO: `/v3/users/3829ae7fdaca46e4b11024008525f47d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:27.856253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3829ae7fdaca46e4b11024008525f47d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:27.858278 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:27.881793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:27.908322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3829ae7fdaca46e4b11024008525f47d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.915936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3829ae7fdaca46e4b11024008525f47d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.935070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3829ae7fdaca46e4b11024008525f47d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.935801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3829ae7fdaca46e4b11024008525f47d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.942998 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-36402cdf-8a73-42bd-a6dc-c8793da17646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3829ae7fdaca46e4b11024008525f47d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:27.949224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1241/2487] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:27 2022] DELETE /identity/v3/users/3829ae7fdaca46e4b11024008525f47d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:28.676529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:28.678212 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:28.695670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa33f9-1bd6-43be-af1c-76607292a20a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:28.766701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:28.767345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:28.767629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:28.767923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] PATH_INFO: `/v3/projects/fb402ba741084c4f855f9ccc08b10229` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:28.769074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb402ba741084c4f855f9ccc08b10229)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:28.771092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:28.823250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb402ba741084c4f855f9ccc08b10229', 'request_id': 'req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:28.827264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:28.829999 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb402ba741084c4f855f9ccc08b10229', 'request_id': 'req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:28.840373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb402ba741084c4f855f9ccc08b10229', 'request_id': 'req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:28.854316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] The token cache is being invalidate because project fb402ba741084c4f855f9ccc08b10229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:28.854711 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:28.855511 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-eb33dcb6-5e2e-422a-8d52-9b243627fe5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:28.860975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1247/2488] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:28 2022] DELETE /identity/v3/projects/fb402ba741084c4f855f9ccc08b10229 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:28.862910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:28.863389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:28.863748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:28.887941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:28.889172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:28.953960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1242/2489] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:28.962690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:28.963639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:28.963925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:28.989136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:28.990392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:28.999590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1248/2490] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:29.003824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.013959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e05c9-b1ce-456c-833e-7ea9f30135d6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.018921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.019513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.019825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.020101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.020795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:29.021105 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.022752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.035118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1243/2491] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:29.039462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.049448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbec79b-2590-485f-80fc-b5f842dc491c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.054117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.054719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.055004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.055279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.055908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.057853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.085395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1249/2492] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:29 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:05:29.089773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.100715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b820c-43ee-4c0b-afd4-b0a1e4616945 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.156671 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.157372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.157657 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.157963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.158595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.160540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.181378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1244/2493] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:29 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:29.185448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.195753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78add237-4e10-47ee-a313-95cb3790699b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.200731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.201330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.201615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.201894 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.202589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.204202 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.216886 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1250/2494] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:29.221603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.235286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902644d5-21b7-424c-a615-5f85cf2eb640 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.241248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.243751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.244253 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.244956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.245827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.248192 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.254549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.256348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:29.263942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1245/2495] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:29.265205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.276934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e85ded-d49b-4b01-825d-19df2e33b5e4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.284010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0aca1-ddde-4625-84ae-e49200be6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.284656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0aca1-ddde-4625-84ae-e49200be6ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.284953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0aca1-ddde-4625-84ae-e49200be6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.285215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0aca1-ddde-4625-84ae-e49200be6ee3 admin admin] PATH_INFO: `/v3/projects/c373cfceb95a4bddb49b9f33c0dbf3ad/users/7e9fd0002671423ba347b75b7c79a0b0/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.287964 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:29.305455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0aca1-ddde-4625-84ae-e49200be6ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c373cfceb95a4bddb49b9f33c0dbf3ad, user_id=7e9fd0002671423ba347b75b7c79a0b0, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.308760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0aca1-ddde-4625-84ae-e49200be6ee3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.324409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1246/2496] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:29 2022] PUT /identity/v3/projects/c373cfceb95a4bddb49b9f33c0dbf3ad/users/7e9fd0002671423ba347b75b7c79a0b0/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:29.334769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.335210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.335633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.350035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd03e8-254f-4487-9f8b-c8329e116118 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.385743 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] MFA Rules not processed for user `7e9fd0002671423ba347b75b7c79a0b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:29.386994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:29.449532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.450378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.450718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.451004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.453941 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:29.462862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1247/2497] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:29.467807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.470488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.474153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.482699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1251/2498] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:29.558909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940bd4c-d3fd-4cb9-be8c-5c87634fe162 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.577869 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.578796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.579206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.579496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.580149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.582167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.611403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1248/2499] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:29 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:29.615998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.629755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932c41a-610e-4b5e-8d23-d07632f4d7c7 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.659419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:29.707847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.709305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.709765 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.710237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.711036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.713969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.745064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1252/2500] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:29 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:29.749124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.762644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce84c0-d0f5-4f86-bdc6-3082e2922dcb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.769886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.770717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.771076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.771426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.772158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.775427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.789006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1249/2501] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:29.792855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.803273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f13272-3d35-44eb-94b7-87f475c713c9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.809478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.810288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.810647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.811066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] PATH_INFO: `/v3/projects/e4c4d4a48b4e4e41b7372ce19d8b7963/users/a83701618fb44cc7add5b18773eb771d/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.828970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c4d4a48b4e4e41b7372ce19d8b7963, user_id=a83701618fb44cc7add5b18773eb771d, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.831970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.848376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1253/2502] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:29 2022] PUT /identity/v3/projects/e4c4d4a48b4e4e41b7372ce19d8b7963/users/a83701618fb44cc7add5b18773eb771d/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:29.852764 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:29.873880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566228b-bfe8-4427-8708-b3f303b085a2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:29.961692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:29.976929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30694a68-4aba-41c1-9c0d-959e87c75d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:29.977538 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30694a68-4aba-41c1-9c0d-959e87c75d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:29.977816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30694a68-4aba-41c1-9c0d-959e87c75d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:29.978116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30694a68-4aba-41c1-9c0d-959e87c75d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:29.978761 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30694a68-4aba-41c1-9c0d-959e87c75d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:29.980348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30694a68-4aba-41c1-9c0d-959e87c75d2b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:29.991594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1250/2503] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:29.995558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:30.006554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3abac2-6d6d-4b49-be46-929273ee8c5d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:30.011701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:30.012367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:30.012821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:30.013097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] PATH_INFO: `/v3/projects/e4c4d4a48b4e4e41b7372ce19d8b7963/users/a83701618fb44cc7add5b18773eb771d/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:30.015726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c4d4a48b4e4e41b7372ce19d8b7963, user_id=a83701618fb44cc7add5b18773eb771d, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:30.018465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:30.035191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1254/2504] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:29 2022] PUT /identity/v3/projects/e4c4d4a48b4e4e41b7372ce19d8b7963/users/a83701618fb44cc7add5b18773eb771d/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:30.046028 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:30.050901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:30.051199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:30.087605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] MFA Rules not processed for user `a83701618fb44cc7add5b18773eb771d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:30.088834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:30.159573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1251/2505] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:30.171691 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:30.173476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:30.235502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a30268-9d65-4886-b4af-d9eef2455568 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:30.239371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:30.240001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:30.240283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:30.240841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:30.252003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:30.253459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:30.263583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1255/2506] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:30.346512 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:30.357495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922085e1-ffaf-4224-8a6c-73410400143d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:30.361663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad4c2-013f-4613-baf0-cfebb39c54e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:30.362391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad4c2-013f-4613-baf0-cfebb39c54e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:30.362610 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad4c2-013f-4613-baf0-cfebb39c54e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:30.362910 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad4c2-013f-4613-baf0-cfebb39c54e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:30.461853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ad4c2-013f-4613-baf0-cfebb39c54e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:30.463366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ad4c2-013f-4613-baf0-cfebb39c54e1 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:30.472622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1252/2507] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:30.747714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:30.819404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0830-b1a2-43a8-85db-d16ed48e6c83 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:30.824086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:30.824717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:30.825001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:30.825282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] PATH_INFO: `/v3/users/7e9fd0002671423ba347b75b7c79a0b0` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:30.826435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9fd0002671423ba347b75b7c79a0b0)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:30.829048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:30.854252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:30.892719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9fd0002671423ba347b75b7c79a0b0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:30.900211 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9fd0002671423ba347b75b7c79a0b0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:30.914234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9fd0002671423ba347b75b7c79a0b0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:30.914919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9fd0002671423ba347b75b7c79a0b0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:30.922518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9fd0002671423ba347b75b7c79a0b0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:30.928631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1256/2508] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:30 2022] DELETE /identity/v3/users/7e9fd0002671423ba347b75b7c79a0b0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:30.933128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:30.933595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:30.933691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:30.953058 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:30.954313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:31.013258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1253/2509] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:31.022583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:31.024014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:31.081897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140b227-0f2a-4831-979e-400b66547b4c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:31.085525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:31.086154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:31.086430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:31.086699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:31.096133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:31.097780 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:31.108832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1257/2510] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:31.387302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:31.388759 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:31.400491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099f253-54f7-473b-bf9e-ab0901eeb010 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:31.405047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:31.406220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:31.406517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:31.406772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] PATH_INFO: `/v3/projects/c373cfceb95a4bddb49b9f33c0dbf3ad` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:31.407888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c373cfceb95a4bddb49b9f33c0dbf3ad)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:31.409796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:31.411596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:31.457321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c373cfceb95a4bddb49b9f33c0dbf3ad', 'request_id': 'req-86e1f1ad-e20d-42fd-bc82-40e4660dafff'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.462984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c373cfceb95a4bddb49b9f33c0dbf3ad', 'request_id': 'req-86e1f1ad-e20d-42fd-bc82-40e4660dafff'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.469848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c373cfceb95a4bddb49b9f33c0dbf3ad', 'request_id': 'req-86e1f1ad-e20d-42fd-bc82-40e4660dafff'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.481862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] The token cache is being invalidate because project c373cfceb95a4bddb49b9f33c0dbf3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:31.482178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.482793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.485033 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1254/2511] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:31 2022] DELETE /identity/v3/projects/c373cfceb95a4bddb49b9f33c0dbf3ad => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:31.489895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:31.499633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501f94-3a66-4f75-abfa-04467d1a6b32 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:31.553802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:31.554376 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:31.554651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:31.554918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] PATH_INFO: `/v3/users/a83701618fb44cc7add5b18773eb771d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:31.556014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83701618fb44cc7add5b18773eb771d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:31.557946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:31.601112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:31.602812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83701618fb44cc7add5b18773eb771d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.612315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83701618fb44cc7add5b18773eb771d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.624264 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83701618fb44cc7add5b18773eb771d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.624898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83701618fb44cc7add5b18773eb771d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.631631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9fe81023-cc99-474b-a383-2779f8ad54d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83701618fb44cc7add5b18773eb771d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:31.637509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1258/2512] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:31 2022] DELETE /identity/v3/users/a83701618fb44cc7add5b18773eb771d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:31.900994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:31.902468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:31.926097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1f1ad-e20d-42fd-bc82-40e4660dafff admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:31.988938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:31.989576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:31.989858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:31.990161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] PATH_INFO: `/v3/projects/e4c4d4a48b4e4e41b7372ce19d8b7963` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:31.991560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c4d4a48b4e4e41b7372ce19d8b7963)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:31.993491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.040788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c4d4a48b4e4e41b7372ce19d8b7963', 'request_id': 'req-83f8e81f-542e-4730-a981-77b6f1481b4a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.047061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c4d4a48b4e4e41b7372ce19d8b7963', 'request_id': 'req-83f8e81f-542e-4730-a981-77b6f1481b4a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.054347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c4d4a48b4e4e41b7372ce19d8b7963', 'request_id': 'req-83f8e81f-542e-4730-a981-77b6f1481b4a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.066938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] The token cache is being invalidate because project e4c4d4a48b4e4e41b7372ce19d8b7963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:32.067225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.067865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-83f8e81f-542e-4730-a981-77b6f1481b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.069377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1255/2513] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:31 2022] DELETE /identity/v3/projects/e4c4d4a48b4e4e41b7372ce19d8b7963 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:32.074134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.074621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.074723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.094358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:32.095466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:32.152734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1259/2514] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:32.161027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.161322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.161602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.181361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:32.182593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:32.190688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1256/2515] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:32.194679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.204147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb2978-e341-47fa-b128-446acbc3a451 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.208635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.209222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.209501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.209770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.210447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:32.210749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.212334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.224097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1260/2516] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:32.228108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.237871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55c12c-d5ad-4504-8670-69260253189e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.242773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.243365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.243648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.243921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.244557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.246672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.259451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:32.279525 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1257/2517] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:32 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:32.283784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.293768 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca69504-6269-4e80-81b9-884c471e9a1a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.353033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.353657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.353960 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.354235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.354898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.356857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.378559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1261/2518] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:32 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:32.382815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.393465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923944a3-f196-47f7-ae7d-e2e97500ec6b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.398190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.398785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.399067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.399343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.399988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.401575 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.412068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1258/2519] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:32.416233 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.427120 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f3bf6-a792-417c-bae5-165ec7b762b4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.431835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5c190-54cc-4135-ada8-06fff7810cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.432430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c190-54cc-4135-ada8-06fff7810cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.432835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c190-54cc-4135-ada8-06fff7810cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.433116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5c190-54cc-4135-ada8-06fff7810cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.433753 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5c190-54cc-4135-ada8-06fff7810cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.435416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5c190-54cc-4135-ada8-06fff7810cbf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.446904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1262/2520] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:05:32.450959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.461140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec8fb6-9b76-474c-905e-40e6dd8150df admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.465921 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.466564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.466849 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.467124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] PATH_INFO: `/v3/projects/953de4e9a1bf4f3d813f121524a5d369/users/eeb2f188bb5d41038a2f0b4ae3ff7fab/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.481817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=953de4e9a1bf4f3d813f121524a5d369, user_id=eeb2f188bb5d41038a2f0b4ae3ff7fab, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.484704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.497523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1259/2521] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:32 2022] PUT /identity/v3/projects/953de4e9a1bf4f3d813f121524a5d369/users/eeb2f188bb5d41038a2f0b4ae3ff7fab/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:05:32.507670 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.507959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.508236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.536647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] MFA Rules not processed for user `eeb2f188bb5d41038a2f0b4ae3ff7fab`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:32.537852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:32.553812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:32.601763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1263/2522] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:32.612867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.614589 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:32.670677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae6561-eb32-4fb5-93af-165ed87e6a9c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.674403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4332a-2175-45d8-ab1c-869478bf1376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.674994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4332a-2175-45d8-ab1c-869478bf1376 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.675279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4332a-2175-45d8-ab1c-869478bf1376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.675553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4332a-2175-45d8-ab1c-869478bf1376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.683628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4332a-2175-45d8-ab1c-869478bf1376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.685047 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4332a-2175-45d8-ab1c-869478bf1376 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.693619 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1260/2523] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:32.712796 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.775661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9eb0d-ddaf-49a4-ab6e-200a5a07c608 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:32.780860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:32.781490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.781772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.782087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] PATH_INFO: `/v3/users/eeb2f188bb5d41038a2f0b4ae3ff7fab` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.783217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb2f188bb5d41038a2f0b4ae3ff7fab)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:32.785135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:32.830658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb2f188bb5d41038a2f0b4ae3ff7fab'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.837970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb2f188bb5d41038a2f0b4ae3ff7fab'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.850217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb2f188bb5d41038a2f0b4ae3ff7fab'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.850896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb2f188bb5d41038a2f0b4ae3ff7fab'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.858290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb2f188bb5d41038a2f0b4ae3ff7fab'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:32.865379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1264/2524] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:32 2022] DELETE /identity/v3/users/eeb2f188bb5d41038a2f0b4ae3ff7fab => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:32.870237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ed0d-0bfd-4412-8367-b584ee9e6b82 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:32.870641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ed0d-0bfd-4412-8367-b584ee9e6b82 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:32.870920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ed0d-0bfd-4412-8367-b584ee9e6b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:32.891934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bec3ed0d-0bfd-4412-8367-b584ee9e6b82 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:32.892947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bec3ed0d-0bfd-4412-8367-b584ee9e6b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:32.952635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1261/2525] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:05:32.962344 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:32.963767 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:33.006595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:33.024144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd59489-f7f0-4bfc-97ca-a8cea109cf67 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:33.027811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:33.028408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.028686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.028983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.038855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:33.040283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:33.050388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1265/2526] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:33.201414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.201897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.202547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.223068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:33.224302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:33.298331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1262/2527] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:05:33.346207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:33.347706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:33.428680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92b48-c8f5-474d-8309-4bec5da81a3d service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:33.435231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8130b92-59c2-4bb1-8ad5-511c022420c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:33.435937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8130b92-59c2-4bb1-8ad5-511c022420c8 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.436390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8130b92-59c2-4bb1-8ad5-511c022420c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.436782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8130b92-59c2-4bb1-8ad5-511c022420c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.446039 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8130b92-59c2-4bb1-8ad5-511c022420c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:33.447809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8130b92-59c2-4bb1-8ad5-511c022420c8 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:33.458163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1266/2528] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:05:33.461713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:33.463299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:33.474831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:33.480798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3e412-9f62-46b6-98b9-aebb47a3f1c4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:33.493429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:33.494403 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.495162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.495768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] PATH_INFO: `/v3/projects/953de4e9a1bf4f3d813f121524a5d369` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.497428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=953de4e9a1bf4f3d813f121524a5d369)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:33.499580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:33.587172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953de4e9a1bf4f3d813f121524a5d369', 'request_id': 'req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:33.594686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953de4e9a1bf4f3d813f121524a5d369', 'request_id': 'req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:33.601767 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953de4e9a1bf4f3d813f121524a5d369', 'request_id': 'req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:33.620494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] The token cache is being invalidate because project 953de4e9a1bf4f3d813f121524a5d369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:33.620717 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:33.621986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7df2ff3-a0eb-47e0-bab7-dddf0ff7a8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:33.625943 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1263/2529] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:33 2022] DELETE /identity/v3/projects/953de4e9a1bf4f3d813f121524a5d369 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:33.634221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.634506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.634798 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.680079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:33.681440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:33.768411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1267/2530] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:33.782578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.783084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.783444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.810799 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:33.813150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:33.823766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1264/2531] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:33.829237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:33.840189 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85be7e-0723-4bf6-8cdc-137282484a07 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:33.846787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:33.847580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.847856 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.848132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.848783 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:33.849087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:33.850737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:33.863885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1268/2532] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:33.868555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:33.879360 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5319f6-423f-487b-8805-51aff1173f51 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:33.886274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:33.887012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:33.887269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:33.887550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:33.888206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:33.890213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:33.933466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1265/2533] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:33 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:33.937499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:33.951113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809ff2-0460-4a45-87b8-bb875cba508e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:34.011051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:34.021956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:34.022832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:34.023501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:34.024142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:34.025064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:34.027192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:34.054946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1269/2534] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:33 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:34.058933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:34.072276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e7489-f7b5-48de-8e36-76e52e3d31c3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:34.078380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:34.079081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:34.079423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:34.079907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:34.080635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:34.082319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:34.096428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1266/2535] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:05:34.100488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:34.112012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2b95f-efef-4b82-9c0f-eef90566c0a1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:34.117506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d0181-5c64-4ea9-8a1b-761b26457047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:34.118136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d0181-5c64-4ea9-8a1b-761b26457047 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:34.118447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d0181-5c64-4ea9-8a1b-761b26457047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:34.118722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d0181-5c64-4ea9-8a1b-761b26457047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:34.119404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d0181-5c64-4ea9-8a1b-761b26457047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:34.121052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d0181-5c64-4ea9-8a1b-761b26457047 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:34.134653 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1270/2536] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:05:34.138771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:34.152130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115eb258-9626-4ad1-80bb-0ad4cb7ff932 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:34.157365 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:34.158319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:34.158606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:34.158885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] PATH_INFO: `/v3/projects/24728534485d4fe98893879d05931ddc/users/7167733ed55649279a73ecef5b5efb17/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:34.175406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24728534485d4fe98893879d05931ddc, user_id=7167733ed55649279a73ecef5b5efb17, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:34.178125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:34.194172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1267/2537] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:34 2022] PUT /identity/v3/projects/24728534485d4fe98893879d05931ddc/users/7167733ed55649279a73ecef5b5efb17/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:34.261354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:34.261688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:34.265008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:34.293468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] MFA Rules not processed for user `7167733ed55649279a73ecef5b5efb17`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:34.294770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:34.385339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1271/2538] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:34.394374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:34.396254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:34.475543 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f783-f53b-4ad5-b1c6-a42ab325e7e3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:34.479569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:34.480188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:34.480472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:34.480748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:34.491535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:34.492979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:34.507129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1268/2539] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:35.374910 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:35.378085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:35.480298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb7c79-00f5-4569-989f-9aac5bcfc9e9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:35.486849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:35.487467 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:35.487751 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:35.488025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:35.489770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:35.503685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:35.505305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6c20e-ed13-4d0b-bc64-cc3e0ccd0c5a service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:35.517271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1272/2540] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:44.500576 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:45.366198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:45.437251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204ab0-9c86-41bd-9c67-f24a25a73dad service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:45.442461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:45.443225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:45.443446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:45.443728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] PATH_INFO: `/v3/users/7167733ed55649279a73ecef5b5efb17` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:45.444911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7167733ed55649279a73ecef5b5efb17)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:45.447274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:45.493850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7167733ed55649279a73ecef5b5efb17'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:45.501098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7167733ed55649279a73ecef5b5efb17'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:45.512252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:45.513386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7167733ed55649279a73ecef5b5efb17'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:45.514729 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7167733ed55649279a73ecef5b5efb17'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:45.522012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7167733ed55649279a73ecef5b5efb17'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:45.527755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1269/2541] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:05:45 2022] DELETE /identity/v3/users/7167733ed55649279a73ecef5b5efb17 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:45.532708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:45.532989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:45.533262 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:45.551475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:45.552641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:45.609407 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1273/2542] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:45.619375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:45.620792 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:45.685480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a331-c1a2-494f-b1c4-160e0ed4dd87 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:45.689583 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc5c69-078d-409b-8d02-43f3f4324920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:45.690211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc5c69-078d-409b-8d02-43f3f4324920 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:45.690489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc5c69-078d-409b-8d02-43f3f4324920 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:45.690757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc5c69-078d-409b-8d02-43f3f4324920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:45.699580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc5c69-078d-409b-8d02-43f3f4324920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:45.700984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc5c69-078d-409b-8d02-43f3f4324920 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:45.710811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1270/2543] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:45.980059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:45.981526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:45.997142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbe99-dd48-4ab1-bb9a-343a77f5adac None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.002414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.003075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.003353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.003627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] PATH_INFO: `/v3/projects/24728534485d4fe98893879d05931ddc` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.004873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] RBAC: Authorizing `identity:delete_project(project_id=24728534485d4fe98893879d05931ddc)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.006699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.055171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24728534485d4fe98893879d05931ddc', 'request_id': 'req-c9559f30-114c-48f2-a50b-29d34b0674df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:46.060645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24728534485d4fe98893879d05931ddc', 'request_id': 'req-c9559f30-114c-48f2-a50b-29d34b0674df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:46.067594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24728534485d4fe98893879d05931ddc', 'request_id': 'req-c9559f30-114c-48f2-a50b-29d34b0674df'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:46.081326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] The token cache is being invalidate because project 24728534485d4fe98893879d05931ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:05:46.081618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:46.082266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c9559f30-114c-48f2-a50b-29d34b0674df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:05:46.083756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1274/2544] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:05:45 2022] DELETE /identity/v3/projects/24728534485d4fe98893879d05931ddc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:46.088785 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.089081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.089366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.109819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:46.111112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:46.130585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:46.176429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1271/2545] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:46.184066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823724f5-88f7-4608-9600-6688407748b0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.184366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823724f5-88f7-4608-9600-6688407748b0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.184648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823724f5-88f7-4608-9600-6688407748b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.205447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-823724f5-88f7-4608-9600-6688407748b0 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:46.206700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-823724f5-88f7-4608-9600-6688407748b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:46.214661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1275/2546] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:46.219061 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.228941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a95448-2504-49c5-9262-0df4f24329b6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.234069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.234706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.234994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.235271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.235922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:05:46.236229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.237825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.249811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1272/2547] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:05:46.254331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823724f5-88f7-4608-9600-6688407748b0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.263412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823724f5-88f7-4608-9600-6688407748b0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.268534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.269121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.269401 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.269667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.270321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.272221 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.298788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1276/2548] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:05:46 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:05:46.302957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.313930 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd122eb1-1d6c-423c-af76-9e22cf6d935b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.368702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.369302 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.369586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.369861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.370514 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.372445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.393325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1273/2549] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:05:46 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:05:46.397387 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.408286 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388d48a-9d0c-4033-a752-495bea6353e0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.412841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.413442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.413740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.414353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.414680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.416276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.427351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1277/2550] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:46.431483 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.441621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe5bcd-c02f-4509-a908-bc4774bb4994 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.446387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97394f9-7d3e-4636-aa55-67c685cb5472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.446987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97394f9-7d3e-4636-aa55-67c685cb5472 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.447274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97394f9-7d3e-4636-aa55-67c685cb5472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.447555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97394f9-7d3e-4636-aa55-67c685cb5472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.448199 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97394f9-7d3e-4636-aa55-67c685cb5472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.449788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97394f9-7d3e-4636-aa55-67c685cb5472 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.460630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1274/2551] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:05:46.464814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.475565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3e33c-8ee7-4998-8a5b-96bb5c4bf435 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.480685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.481443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.481821 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.482319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] PATH_INFO: `/v3/projects/485fee8f1549466aa2b477335ae64453/users/d0b512ff1ce040f2b18c11d4c58ddca3/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.501266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485fee8f1549466aa2b477335ae64453, user_id=d0b512ff1ce040f2b18c11d4c58ddca3, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.504016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.519421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1278/2552] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:05:46 2022] PUT /identity/v3/projects/485fee8f1549466aa2b477335ae64453/users/d0b512ff1ce040f2b18c11d4c58ddca3/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:05:46.532700 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8fae-ecf2-454b-bdca-69f934fbd7d4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.533588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8fae-ecf2-454b-bdca-69f934fbd7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.533871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8fae-ecf2-454b-bdca-69f934fbd7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.565868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f73c8fae-ecf2-454b-bdca-69f934fbd7d4 None None] MFA Rules not processed for user `d0b512ff1ce040f2b18c11d4c58ddca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:46.567218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f73c8fae-ecf2-454b-bdca-69f934fbd7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:46.653816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1275/2553] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:46.665874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:46.668444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:46.696275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:46.738382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fc0a2-f323-4acf-8f29-1810996cf874 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:46.742848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a63b2b-55b1-4253-9a0c-2dc8bc90f16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:46.743627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a63b2b-55b1-4253-9a0c-2dc8bc90f16b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:46.744088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a63b2b-55b1-4253-9a0c-2dc8bc90f16b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:46.744476 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a63b2b-55b1-4253-9a0c-2dc8bc90f16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:46.754105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a63b2b-55b1-4253-9a0c-2dc8bc90f16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:46.755548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a63b2b-55b1-4253-9a0c-2dc8bc90f16b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:46.764923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1279/2554] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:47.881446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c3f52-fddc-4f01-b8c5-acb9de28af88 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:47.882055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c3f52-fddc-4f01-b8c5-acb9de28af88 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:47.882055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c3f52-fddc-4f01-b8c5-acb9de28af88 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:47.883384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1276/2555] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:05:47.889555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc2ecc-383e-406b-9ac7-3466ad136fd8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:47.890043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc2ecc-383e-406b-9ac7-3466ad136fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:47.890151 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc2ecc-383e-406b-9ac7-3466ad136fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:47.910462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-47bc2ecc-383e-406b-9ac7-3466ad136fd8 None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:47.911659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-47bc2ecc-383e-406b-9ac7-3466ad136fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:47.985693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1280/2556] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:47.999240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c3f52-fddc-4f01-b8c5-acb9de28af88 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:48.001229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-948c3f52-fddc-4f01-b8c5-acb9de28af88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:48.023495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:05:48.062933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c3f52-fddc-4f01-b8c5-acb9de28af88 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:48.066440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d84bc-0940-49ce-880d-9549fa35f445 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:48.067041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d84bc-0940-49ce-880d-9549fa35f445 service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.067334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d84bc-0940-49ce-880d-9549fa35f445 service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.067611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d84bc-0940-49ce-880d-9549fa35f445 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.078345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d84bc-0940-49ce-880d-9549fa35f445 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:48.079774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d84bc-0940-49ce-880d-9549fa35f445 service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:48.090411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1277/2557] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:48.158834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543baf-35a3-4814-94e6-3031a31c3eb9 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.159121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543baf-35a3-4814-94e6-3031a31c3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.159392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543baf-35a3-4814-94e6-3031a31c3eb9 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.160975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1281/2558] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:05:48.165549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996a619-6e9c-44b2-8715-bbfa1369bf4b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.165840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996a619-6e9c-44b2-8715-bbfa1369bf4b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.166138 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996a619-6e9c-44b2-8715-bbfa1369bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.188109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8996a619-6e9c-44b2-8715-bbfa1369bf4b None None] MFA Rules not processed for user `84a66b2803614363a196ecd94828a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:48.189334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8996a619-6e9c-44b2-8715-bbfa1369bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:48.247663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1278/2559] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:05:48.260203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543baf-35a3-4814-94e6-3031a31c3eb9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:48.261885 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6d543baf-35a3-4814-94e6-3031a31c3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:48.278563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543baf-35a3-4814-94e6-3031a31c3eb9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:48.283074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc09b6f-2f1d-4d56-86aa-29004e1f07be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:48.286651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09b6f-2f1d-4d56-86aa-29004e1f07be service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.287028 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09b6f-2f1d-4d56-86aa-29004e1f07be service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.287371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09b6f-2f1d-4d56-86aa-29004e1f07be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.296317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc09b6f-2f1d-4d56-86aa-29004e1f07be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:48.298196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc09b6f-2f1d-4d56-86aa-29004e1f07be service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:48.307926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1282/2560] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:48.396439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401e4a6-8445-4789-98f8-ca58c66864d1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.396736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401e4a6-8445-4789-98f8-ca58c66864d1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.397010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401e4a6-8445-4789-98f8-ca58c66864d1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.398412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1279/2561] 10.222.0.61 () {54 vars in 921 bytes} [Sat Sep 24 15:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 15:05:48.403682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf96c8-ce45-4fb6-9cc5-3e4853c6ee73 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.403973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf96c8-ce45-4fb6-9cc5-3e4853c6ee73 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.404255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf96c8-ce45-4fb6-9cc5-3e4853c6ee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.424651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ebaf96c8-ce45-4fb6-9cc5-3e4853c6ee73 None None] MFA Rules not processed for user `dc85d332eeaf4c649f51858fa608101f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:48.425853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ebaf96c8-ce45-4fb6-9cc5-3e4853c6ee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:48.435062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1283/2562] 10.222.0.61 () {58 vars in 1018 bytes} [Sat Sep 24 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:48.445376 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401e4a6-8445-4789-98f8-ca58c66864d1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:48.456704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401e4a6-8445-4789-98f8-ca58c66864d1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:48.460254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:48.460865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:48.461150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:48.461425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:48.471827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:48.473231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:48.483593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1280/2563] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:05:55.905083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:55.905672 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:55.905777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:55.928583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:05:55.929794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:55.987243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1284/2564] 10.222.0.61 () {58 vars in 1009 bytes} [Sat Sep 24 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:05:56.000378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:05:56.001797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:05:56.071692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1945159-3d5b-441f-82bb-6d2e86788cb8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:05:56.076739 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600a1f5-9b99-4e63-9ff0-2fd87f87b16d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:05:56.077338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600a1f5-9b99-4e63-9ff0-2fd87f87b16d service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:05:56.077638 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600a1f5-9b99-4e63-9ff0-2fd87f87b16d service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:05:56.078696 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600a1f5-9b99-4e63-9ff0-2fd87f87b16d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:05:56.086751 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600a1f5-9b99-4e63-9ff0-2fd87f87b16d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:05:56.088345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600a1f5-9b99-4e63-9ff0-2fd87f87b16d service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:05:56.097385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1281/2565] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:05:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:06:05.197087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:05.240529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:05.264926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf71a2-ccb1-463d-bee3-3be92e56b8a1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:05.269507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:05.270170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:05.270485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:05.270761 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] PATH_INFO: `/v3/users/d0b512ff1ce040f2b18c11d4c58ddca3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:05.271925 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b512ff1ce040f2b18c11d4c58ddca3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:05.273855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:05.322053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b512ff1ce040f2b18c11d4c58ddca3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:05.333077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b512ff1ce040f2b18c11d4c58ddca3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:05.347390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b512ff1ce040f2b18c11d4c58ddca3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:05.348130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b512ff1ce040f2b18c11d4c58ddca3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:05.355432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b512ff1ce040f2b18c11d4c58ddca3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:05.362031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1285/2566] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:06:05 2022] DELETE /identity/v3/users/d0b512ff1ce040f2b18c11d4c58ddca3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:05.367088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:05.367652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:05.367652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:05.387336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:05.388559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:05.448796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:05.472725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1282/2567] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:05.482833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:05.484379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:05.546466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2d4a-6bd2-4265-8a59-01fbad4321d6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:05.550758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5a067-f5fa-4962-b83d-6affa09974a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:05.551389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a067-f5fa-4962-b83d-6affa09974a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:05.551674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a067-f5fa-4962-b83d-6affa09974a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:05.552181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a067-f5fa-4962-b83d-6affa09974a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:05.563399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5a067-f5fa-4962-b83d-6affa09974a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:05.564862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5a067-f5fa-4962-b83d-6affa09974a7 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:05.575391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1286/2568] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:06.023232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:06.025163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:06.040366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1ea30-eb60-41fe-a638-ad479c2efe03 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:06.046126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:06.048272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:06.048553 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:06.048824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] PATH_INFO: `/v3/projects/485fee8f1549466aa2b477335ae64453` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:06.050071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485fee8f1549466aa2b477335ae64453)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:06.051997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:06.105572 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485fee8f1549466aa2b477335ae64453', 'request_id': 'req-d7bf62ae-e355-4142-bce8-4b36de443f22'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:06.112193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485fee8f1549466aa2b477335ae64453', 'request_id': 'req-d7bf62ae-e355-4142-bce8-4b36de443f22'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:06.118736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485fee8f1549466aa2b477335ae64453', 'request_id': 'req-d7bf62ae-e355-4142-bce8-4b36de443f22'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:06.132270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] The token cache is being invalidate because project 485fee8f1549466aa2b477335ae64453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:06:06.132741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:06.133402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:06.135036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1283/2569] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:06:06 2022] DELETE /identity/v3/projects/485fee8f1549466aa2b477335ae64453 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:13.975174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:13.975668 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:13.975789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:13.997696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:13.999076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:14.056783 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1287/2570] 10.222.0.61 () {58 vars in 967 bytes} [Sat Sep 24 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:14.069846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:14.071322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:14.143678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf62ae-e355-4142-bce8-4b36de443f22 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:14.151130 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ee82-b872-4abb-a199-fda371f46aa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:14.151878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ee82-b872-4abb-a199-fda371f46aa9 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:14.152173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ee82-b872-4abb-a199-fda371f46aa9 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:14.152451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ee82-b872-4abb-a199-fda371f46aa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:14.166305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ee82-b872-4abb-a199-fda371f46aa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:14.167981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ee82-b872-4abb-a199-fda371f46aa9 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:14.178836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1284/2571] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:06:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 15:06:24.049453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:24.175009 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=65,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:26.036130 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:26.037606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:26.122026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10f272-8efa-4b1a-938c-c236d8bf2b5c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:26.131022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:26.131428 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:26.131705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:26.131977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:26.133943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:26.147246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:26.148858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:26.175921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1288/2572] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:35.829070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:35.829536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:35.829630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:35.849602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] MFA Rules not processed for user `82bd05cad09d4763a79fc7400921c3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:35.850888 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:35.858160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1285/2573] 10.222.0.61 () {58 vars in 967 bytes} [Sat Sep 24 15:06:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:35.880430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:35.891738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff8d2c-d92f-42f0-8280-3c4c66f15c2e service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:35.896876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756ac401fea141e68343168d6213d2b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:35.897465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:35.897743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:35.898046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:35.906338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:35.907931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:35.915683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1289/2574] 10.222.0.61 () {60 vars in 1384 bytes} [Sat Sep 24 15:06:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 15:06:39.183942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:39.185280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:39.267460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804e771-a103-4453-bf0c-e1b36d8906e1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:39.273420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:39.274096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:39.274388 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:39.274780 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:39.276504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:39.289510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:39.291154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:39.302778 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1286/2575] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:41.045131 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:41.047198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:41.107724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8c74b-b7cc-41b7-b2bc-60c8ee84f613 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:41.111612 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:41.112327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:41.112699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:41.113066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:41.114957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:41.182225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:41.184328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:41.195455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1290/2576] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:42.615055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:42.617699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:42.632738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810906e-eb46-4416-924b-830ef5013e1c service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:42.639123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:42.639893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:42.640172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:42.640464 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] PATH_INFO: `/v3/users/4e9a50200076443d98841a76e961408b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:42.641691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9a50200076443d98841a76e961408b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:42.643774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:42.708683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9a50200076443d98841a76e961408b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:42.719013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9a50200076443d98841a76e961408b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:42.733845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9a50200076443d98841a76e961408b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:42.734630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9a50200076443d98841a76e961408b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:42.741815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9a50200076443d98841a76e961408b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:42.748175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1287/2577] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:06:42 2022] DELETE /identity/v3/users/4e9a50200076443d98841a76e961408b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:42.927712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:42.941862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db2904-81b3-4ce6-9892-95d3fd466ce9 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:42.980864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:43.052423 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:43.053820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.054306 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.054723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] PATH_INFO: `/v3/projects/1afc2cc56fef4d5aa0e0905ef059d123` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.056569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1afc2cc56fef4d5aa0e0905ef059d123)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:43.060369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:43.123260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1afc2cc56fef4d5aa0e0905ef059d123', 'request_id': 'req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.129838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1afc2cc56fef4d5aa0e0905ef059d123', 'request_id': 'req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.136859 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1afc2cc56fef4d5aa0e0905ef059d123', 'request_id': 'req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.153968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] The token cache is being invalidate because project 1afc2cc56fef4d5aa0e0905ef059d123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:06:43.154339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.155023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.162282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1291/2578] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:06:42 2022] DELETE /identity/v3/projects/1afc2cc56fef4d5aa0e0905ef059d123 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:43.163584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:43.165829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:43.182686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95b3ef-32c0-43f6-9f7b-d815aeea25aa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:43.238054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:43.248176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:43.249854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.250381 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.250874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] PATH_INFO: `/v3/users/2ff5db15af92475aa6219d9e28914652` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.252733 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ff5db15af92475aa6219d9e28914652)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:43.255358 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:43.309007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ff5db15af92475aa6219d9e28914652'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.318644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ff5db15af92475aa6219d9e28914652'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.334143 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ff5db15af92475aa6219d9e28914652'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.335241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ff5db15af92475aa6219d9e28914652'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.352763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3a1b48c-f31a-48f7-b60a-b700c35b099e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ff5db15af92475aa6219d9e28914652'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.362018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1288/2579] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:06:43 2022] DELETE /identity/v3/users/2ff5db15af92475aa6219d9e28914652 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:43.564644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:43.566245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:43.593291 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfea4d-cf9b-42e5-a9ab-4d2870a32fbe admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:43.656673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:43.657421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.657710 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.658012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] PATH_INFO: `/v3/projects/555986b504ca4282bcee2de77528f1ba` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.659148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=555986b504ca4282bcee2de77528f1ba)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:43.661089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:43.711072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555986b504ca4282bcee2de77528f1ba', 'request_id': 'req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.715509 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:43.719452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555986b504ca4282bcee2de77528f1ba', 'request_id': 'req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.726057 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555986b504ca4282bcee2de77528f1ba', 'request_id': 'req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.742301 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] The token cache is being invalidate because project 555986b504ca4282bcee2de77528f1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:06:43.742792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.744046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5ad4d541-7c8b-4b36-9f4d-d84c67cc9104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:43.750250 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1292/2580] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:06:43 2022] DELETE /identity/v3/projects/555986b504ca4282bcee2de77528f1ba => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:43.753111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.753603 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.753720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.784931 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:43.786335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:43.850390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1289/2581] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:43.859803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.860091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.860369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.880595 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:43.881809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:43.890439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1293/2582] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:43.894704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:43.904456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423af0a-0d04-45b4-aa2d-277132daaf0e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:43.909863 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:43.910634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.910918 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.911216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.911866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:06:43.912184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:43.913760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:43.926193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1290/2583] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:06:43.930700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:43.940184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532e01a-48b1-46e4-965f-7b58b10ca1e0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:43.944833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:43.945469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:43.945896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:43.946472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:43.947227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:43.949126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:43.981685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1294/2584] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:06:43 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:06:43.982843 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:43.993969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f313e8d-1786-4c31-b410-0bf44bda17dc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:44.063247 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:44.063865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.064149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.064427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.065084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:44.067067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:44.076803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:44.098155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1291/2585] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:06:43 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:44.099054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:44.109603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df5ee-1746-4bad-b40d-c54123ec5990 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:44.114452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:44.115067 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.115353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.116146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.116265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:44.118433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:44.130037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1295/2586] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:06:44.134420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:44.146238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69827a07-e12b-4e20-8b71-529a789f762a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:44.151318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e45a0-9411-4f2b-a8ed-c15e404620ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:44.152111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e45a0-9411-4f2b-a8ed-c15e404620ee admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.153170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e45a0-9411-4f2b-a8ed-c15e404620ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.153170 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e45a0-9411-4f2b-a8ed-c15e404620ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.153947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e45a0-9411-4f2b-a8ed-c15e404620ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:44.155652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e45a0-9411-4f2b-a8ed-c15e404620ee admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:44.169979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1292/2587] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:06:44.176905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:44.189411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598a5b0-e2db-4a69-ba80-4cb84fb31cb8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:44.194936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:44.195831 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.196115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.196394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] PATH_INFO: `/v3/projects/cd72399e0d4f4e25b992e8e8cf836c18/users/5a93a2d9266541e7bff502ba074fef5c/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.215664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd72399e0d4f4e25b992e8e8cf836c18, user_id=5a93a2d9266541e7bff502ba074fef5c, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:44.218822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:44.242243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1296/2588] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:44 2022] PUT /identity/v3/projects/cd72399e0d4f4e25b992e8e8cf836c18/users/5a93a2d9266541e7bff502ba074fef5c/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:44.247240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.247647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.248075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.269189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:44.270520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:44.334715 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1293/2589] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:44.344465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:44.346190 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:44.405259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ecc97-538b-4de1-b0f1-b2852069586d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:44.408964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:44.409572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.409852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.410158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.419234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:44.421501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:44.433156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1297/2590] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:44.720627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:44.722129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:44.738456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d608-8443-43f4-86e6-76218503794c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:44.744722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:44.746141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:44.746432 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:44.746816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] PATH_INFO: `/v3/users/9f74f51060554db18bc96199222e9261` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:44.748091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f74f51060554db18bc96199222e9261)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:44.750131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:44.806816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f74f51060554db18bc96199222e9261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:44.815842 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f74f51060554db18bc96199222e9261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:44.833841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f74f51060554db18bc96199222e9261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:44.835205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f74f51060554db18bc96199222e9261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:44.843681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-858c3f67-8e00-4f53-8ef6-e832e670c430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f74f51060554db18bc96199222e9261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:44.850681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1294/2591] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:06:44 2022] DELETE /identity/v3/users/9f74f51060554db18bc96199222e9261 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:45.178823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.189946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b278f-d31e-468e-a60b-d3efa20de04f service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.279144 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.279601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.280032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.280457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] PATH_INFO: `/v3/projects/9b2919ee12fe488ea94db84e9af057a2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.282342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b2919ee12fe488ea94db84e9af057a2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.284020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:45.314517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:45.347225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b2919ee12fe488ea94db84e9af057a2', 'request_id': 'req-dffb67be-cd20-4a70-8a41-19dc7d16a76c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:45.356229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b2919ee12fe488ea94db84e9af057a2', 'request_id': 'req-dffb67be-cd20-4a70-8a41-19dc7d16a76c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:45.364822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b2919ee12fe488ea94db84e9af057a2', 'request_id': 'req-dffb67be-cd20-4a70-8a41-19dc7d16a76c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:45.382251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] The token cache is being invalidate because project 9b2919ee12fe488ea94db84e9af057a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:06:45.382587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:45.383271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dffb67be-cd20-4a70-8a41-19dc7d16a76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:45.385203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1298/2592] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:06:45 2022] DELETE /identity/v3/projects/9b2919ee12fe488ea94db84e9af057a2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:45.390972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.391255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.391716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.414497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:45.415919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:45.503870 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:45.526517 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1295/2593] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:06:45.528495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.528786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.529065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.556102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:45.557453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:45.573955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1299/2594] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:45.578529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.595164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76718f-e281-4aa1-927c-559661718fdb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.600078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.600690 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.600975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.601252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.601926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:06:45.602264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.604000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:45.623455 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1296/2595] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:06:45.626656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.642583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4846aaa-64e7-4414-acaa-8649cc2d92dd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.648647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.649996 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.650242 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.650495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.651147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.653099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:45.690623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1300/2596] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:06:45 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:45.692905 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.707054 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f914fb-8723-46d3-adba-2db9d02f14db admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.789409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.790284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.790788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.791222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.792078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.794046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:45.828795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1297/2597] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:06:45 2022] POST /identity/v3/users => generated 283 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:06:45.830053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.870398 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe0be5-89f5-4622-9d7d-b110e109671e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.875099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.876052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.876052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.876274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.877089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.878745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:45.890982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1301/2598] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:06:45.894972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.907430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f15fb-39ac-410d-89e6-e9f88c7c553e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.912069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e9300-dade-4d67-9893-cdecf14a67ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.912678 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e9300-dade-4d67-9893-cdecf14a67ba admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.913210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e9300-dade-4d67-9893-cdecf14a67ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.913489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e9300-dade-4d67-9893-cdecf14a67ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.918718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e9300-dade-4d67-9893-cdecf14a67ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.921373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e9300-dade-4d67-9893-cdecf14a67ba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:45.935504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1298/2599] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:06:45.939371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:45.954257 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53efcf-1a9f-469b-b66f-6b5958fa3ff8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:45.962149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:45.962966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:45.963457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:45.963730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] PATH_INFO: `/v3/projects/2bab060a18374deb89c64d80c1b3bdd3/users/458b376303df45a28c331a5145b7977e/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:45.981358 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bab060a18374deb89c64d80c1b3bdd3, user_id=458b376303df45a28c331a5145b7977e, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:45.984196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.009958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1302/2600] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:45 2022] PUT /identity/v3/projects/2bab060a18374deb89c64d80c1b3bdd3/users/458b376303df45a28c331a5145b7977e/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:46.016348 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.018015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.018680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.059231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] MFA Rules not processed for user `458b376303df45a28c331a5145b7977e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:46.060775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:46.151256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1299/2601] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:46.156491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.260804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbd922-2cfa-4f6c-8501-62eb3efe918e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.267323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.268011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.268353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.268692 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.269443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.271468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.307874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1303/2602] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:06:46 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:46.312794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.325079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ef5a6-34b6-4319-ac1d-5bf4439755e6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.379871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:46.397516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.398175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.399065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.399065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.399357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.401737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.427871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1300/2603] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:06:46 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:46.429141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.440297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c085be0-71eb-4e1d-8486-04d5fac53dee admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.445134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.445758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.446208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.446484 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.447155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.448750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.461440 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1304/2604] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:06:46.465755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.475811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7fc1c-315b-432e-94ea-863b502e44a0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.480544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.481258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.481592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.481828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] PATH_INFO: `/v3/projects/cca5ede40c8b4a809f506b30eb8af841/users/fb7ae6a297ee45edb4a6cb16693b9810/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.497682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca5ede40c8b4a809f506b30eb8af841, user_id=fb7ae6a297ee45edb4a6cb16693b9810, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.500972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.515082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1301/2605] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:46 2022] PUT /identity/v3/projects/cca5ede40c8b4a809f506b30eb8af841/users/fb7ae6a297ee45edb4a6cb16693b9810/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:46.520132 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.530274 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e27c52-9cad-4b58-a3d2-9652acb632f1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.548822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:46.591770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb4c4-885f-4c04-b18f-664731f8183c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.592847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb4c4-885f-4c04-b18f-664731f8183c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.592847 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb4c4-885f-4c04-b18f-664731f8183c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.593076 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb4c4-885f-4c04-b18f-664731f8183c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.593665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb4c4-885f-4c04-b18f-664731f8183c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.595491 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb4c4-885f-4c04-b18f-664731f8183c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.605294 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1305/2606] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:06:46.609624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.620084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37039b4-e5cf-4c31-98da-075ca8f22ffb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.625188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.625833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.626153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.626438 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] PATH_INFO: `/v3/projects/cca5ede40c8b4a809f506b30eb8af841/users/fb7ae6a297ee45edb4a6cb16693b9810/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.628391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca5ede40c8b4a809f506b30eb8af841, user_id=fb7ae6a297ee45edb4a6cb16693b9810, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.631095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.642691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1302/2607] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:46 2022] PUT /identity/v3/projects/cca5ede40c8b4a809f506b30eb8af841/users/fb7ae6a297ee45edb4a6cb16693b9810/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:46.653890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.654210 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.654482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.682331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] MFA Rules not processed for user `fb7ae6a297ee45edb4a6cb16693b9810`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:46.683536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:46.765791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1306/2608] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:46.778908 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.780793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:46.850113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf54d2-08b8-4d1b-b853-0dd672411a7c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:46.854164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd66aa3-49bf-4ecb-8045-3078f20dc031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:46.854889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd66aa3-49bf-4ecb-8045-3078f20dc031 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:46.855245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd66aa3-49bf-4ecb-8045-3078f20dc031 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:46.855595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd66aa3-49bf-4ecb-8045-3078f20dc031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:46.868126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd66aa3-49bf-4ecb-8045-3078f20dc031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:46.869745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd66aa3-49bf-4ecb-8045-3078f20dc031 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:46.881673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1303/2609] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:46.987315 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:46.999832 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f523fe0-2cd0-42f6-9a08-6d4107f2f2b6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:47.003493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:47.004099 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:47.004377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:47.004646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:47.087314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:47.088937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:47.100589 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1307/2610] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:49.428396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.429240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.429532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.462250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] MFA Rules not processed for user `5a93a2d9266541e7bff502ba074fef5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:49.463523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:49.538608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1304/2611] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:49.543088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:49.544556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:49.597687 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:49.626586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338d4d4-174d-47fb-9c4e-aeb87217bd6e service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:49.631316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:49.632000 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.632289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.632565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.633408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:49.635481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:49.664981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1308/2612] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:06:49 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:49.669015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:49.679692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea2628-dfc8-4ac4-a083-ad329e2f016c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:49.710974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:49.738493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:49.739103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.739411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.739686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.740315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:49.742523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:49.763074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1305/2613] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:06:49 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:49.767561 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:49.777640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad4a5d-ff03-484d-8b6e-87e34494d889 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:49.782841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:49.783427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.783703 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.783973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.784656 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:49.786231 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:49.796594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1309/2614] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:06:49.800797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:49.810559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed83aa-9ced-400f-965a-6643f458ad68 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:49.815103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:49.815734 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.816013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.816286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] PATH_INFO: `/v3/projects/3c166880b2924c868d69e3abc7e03e7f/users/e3ca8f7b57314a2484f4deeb7caea50a/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.831043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c166880b2924c868d69e3abc7e03e7f, user_id=e3ca8f7b57314a2484f4deeb7caea50a, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:49.833655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:49.847330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1306/2615] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:49 2022] PUT /identity/v3/projects/3c166880b2924c868d69e3abc7e03e7f/users/e3ca8f7b57314a2484f4deeb7caea50a/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:49.851296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:49.862699 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4eaa5-9a91-453d-b50c-68b4facfedd3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:49.923044 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087964f5-3557-40db-a2b0-9f8d388bd72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:49.923664 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087964f5-3557-40db-a2b0-9f8d388bd72f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.923947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087964f5-3557-40db-a2b0-9f8d388bd72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.924223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087964f5-3557-40db-a2b0-9f8d388bd72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.924865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087964f5-3557-40db-a2b0-9f8d388bd72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:49.926505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087964f5-3557-40db-a2b0-9f8d388bd72f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:49.938273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1310/2616] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:06:49.942503 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:49.953288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eca9f5-c6ce-49be-94a8-35a50565112b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:49.958223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:49.958872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:49.959162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:49.959441 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] PATH_INFO: `/v3/projects/3c166880b2924c868d69e3abc7e03e7f/users/e3ca8f7b57314a2484f4deeb7caea50a/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:49.961361 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c166880b2924c868d69e3abc7e03e7f, user_id=e3ca8f7b57314a2484f4deeb7caea50a, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:49.964038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:49.975085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1307/2617] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:49 2022] PUT /identity/v3/projects/3c166880b2924c868d69e3abc7e03e7f/users/e3ca8f7b57314a2484f4deeb7caea50a/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:55.875156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:55.875983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:55.876546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:55.929235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] MFA Rules not processed for user `e3ca8f7b57314a2484f4deeb7caea50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:55.930584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:56.032736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1311/2618] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:56.047238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:56.049530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:56.147404 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c01558-2ad3-4e8a-b4b6-bb8e06c12738 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:56.156319 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03b696-b7a2-4386-9741-dcdd77ab1324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:56.157790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b696-b7a2-4386-9741-dcdd77ab1324 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:56.164497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b696-b7a2-4386-9741-dcdd77ab1324 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:56.164991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03b696-b7a2-4386-9741-dcdd77ab1324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:56.275631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03b696-b7a2-4386-9741-dcdd77ab1324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:56.279374 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03b696-b7a2-4386-9741-dcdd77ab1324 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:56.294959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1308/2619] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:06:57.277866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:57.344421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:57.372289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455928c7-0376-4f9d-8f8d-261664f9275b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:57.383077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:57.383747 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:57.384034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:57.384313 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] PATH_INFO: `/v3/users/458b376303df45a28c331a5145b7977e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:57.385557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458b376303df45a28c331a5145b7977e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:57.388112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:57.449046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458b376303df45a28c331a5145b7977e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:57.459271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458b376303df45a28c331a5145b7977e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:57.475848 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458b376303df45a28c331a5145b7977e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:57.478298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458b376303df45a28c331a5145b7977e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:57.486006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458b376303df45a28c331a5145b7977e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:57.494565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1312/2620] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:06:57 2022] DELETE /identity/v3/users/458b376303df45a28c331a5145b7977e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:57.497835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:57.499248 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:57.499551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:57.522352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:57.523879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:57.550249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:57.605649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1309/2621] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:57.617458 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:57.619089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:57.692658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcb745-a121-4c84-be2f-7fbebab6c160 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:57.696379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:57.697133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:57.697554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:57.697942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:57.709234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:57.711025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:57.724722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1313/2622] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:06:58.084594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:58.087340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:58.105284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e93e21-9792-477f-ab23-7e874f447776 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:58.129111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:58.131046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:58.133493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:58.134086 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] PATH_INFO: `/v3/projects/2bab060a18374deb89c64d80c1b3bdd3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:58.135957 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bab060a18374deb89c64d80c1b3bdd3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:58.140498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:58.247124 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bab060a18374deb89c64d80c1b3bdd3', 'request_id': 'req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.259353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bab060a18374deb89c64d80c1b3bdd3', 'request_id': 'req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.269387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bab060a18374deb89c64d80c1b3bdd3', 'request_id': 'req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.303506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] The token cache is being invalidate because project 2bab060a18374deb89c64d80c1b3bdd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:06:58.303991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.306871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.312236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1310/2623] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:06:58 2022] DELETE /identity/v3/projects/2bab060a18374deb89c64d80c1b3bdd3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:58.316426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:58.331349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c3ae-f6e1-4457-8bb7-97fa1ed65d75 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:58.399330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:58.400179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:58.400562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:58.401038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] PATH_INFO: `/v3/users/fb7ae6a297ee45edb4a6cb16693b9810` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:58.402380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7ae6a297ee45edb4a6cb16693b9810)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:58.404453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:58.406690 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:58.469521 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7ae6a297ee45edb4a6cb16693b9810'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.488248 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7ae6a297ee45edb4a6cb16693b9810'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.501997 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7ae6a297ee45edb4a6cb16693b9810'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.502674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7ae6a297ee45edb4a6cb16693b9810'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.517380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8a8337c4-9e7e-4153-9340-ff7e2255bf27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7ae6a297ee45edb4a6cb16693b9810'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:58.525299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1314/2624] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:06:58 2022] DELETE /identity/v3/users/fb7ae6a297ee45edb4a6cb16693b9810 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:58.875300 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:58.877226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:58.899077 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2e0d4-65fc-47e3-8c3e-2799b03e4800 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:58.982569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:58.983258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:58.983540 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:58.983814 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] PATH_INFO: `/v3/projects/cca5ede40c8b4a809f506b30eb8af841` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:58.986798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca5ede40c8b4a809f506b30eb8af841)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:58.988779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.042465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:59.046633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca5ede40c8b4a809f506b30eb8af841', 'request_id': 'req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:59.054504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca5ede40c8b4a809f506b30eb8af841', 'request_id': 'req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:59.061224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca5ede40c8b4a809f506b30eb8af841', 'request_id': 'req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:59.076026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] The token cache is being invalidate because project cca5ede40c8b4a809f506b30eb8af841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:06:59.076322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:59.077068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98b57aa0-68cf-48b3-8e0b-7dd2c4e64a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:06:59.078860 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1311/2625] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:06:58 2022] DELETE /identity/v3/projects/cca5ede40c8b4a809f506b30eb8af841 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:06:59.083380 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.083673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.083944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.106001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:59.109364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:59.177515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1315/2626] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:59.185321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.185640 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.186477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.207264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:59.208471 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:59.216357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1312/2627] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:59.221014 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.231124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c65f0-660c-4ed8-9f42-9e4e777a4f37 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.235694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.236299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.236581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.236857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.237498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:06:59.237804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.239436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.252600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1316/2628] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:06:59.256883 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.266867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27be246-5ca3-4ddc-bb4a-405c57a6fe8f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.271425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.272043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.272330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.272781 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.273486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.275382 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.305444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1313/2629] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:06:59.307348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.318617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5c966-f7c8-4e9a-9860-ea5d8f017377 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.401360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.402435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.402435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.404921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.404921 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.405513 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.417600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:06:59.433978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1317/2630] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/users => generated 303 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:59.441261 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.446549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.448464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:59.451991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1652ce-ea69-4eb0-b84e-7973d8946d5a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.456960 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.457566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.457845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.458146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.458790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.460385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.470738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1314/2631] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:06:59.474867 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.486184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c159b6a-711d-40f5-92d6-4e9c9479e989 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.491020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.491882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.492168 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.492472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.493251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.494902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.507027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1315/2632] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:06:59.511049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.513022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95274c34-fe36-4ed2-92e2-70eaa9cac55d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.516794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.517433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.517717 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.518020 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.519898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:59.521450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68ce22-fecd-4bd1-a468-4e7f09d0f41d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.526171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.526826 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.527114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.527383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] PATH_INFO: `/v3/projects/7816f3fa6d424d48ac2dea3dda45b0e3/users/18bceb4905ac4d799a93e60d3b684180/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.533126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.534634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461f15a-a4d3-4a21-b034-804be996ac6b service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.542779 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7816f3fa6d424d48ac2dea3dda45b0e3, user_id=18bceb4905ac4d799a93e60d3b684180, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.544457 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1318/2633] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:06:59.546016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.560590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1316/2634] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:59 2022] PUT /identity/v3/projects/7816f3fa6d424d48ac2dea3dda45b0e3/users/18bceb4905ac4d799a93e60d3b684180/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:06:59.572789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.573080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.573361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.602613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] MFA Rules not processed for user `18bceb4905ac4d799a93e60d3b684180`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:06:59.603849 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:06:59.678754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1319/2635] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:06:59.683313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.769434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fceffe-ccec-4997-9639-64895e25608c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.774588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.775391 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.775673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.775948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.776648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.778681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.816216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1317/2636] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/projects => generated 407 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:06:59.820235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.832246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab80da4-7a52-46e6-9854-c7a5e5e1ba11 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.898584 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.899254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.899540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.901180 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.901855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.903932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.942808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1320/2637] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:06:59 2022] POST /identity/v3/users => generated 310 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:06:59.944107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.958125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd876-165f-4f90-99fe-dcab07a00aa5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:06:59.963451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:06:59.964080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:06:59.964417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:06:59.964699 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:06:59.965375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:06:59.967072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:06:59.981068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1318/2638] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:06:59.984902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:06:59.996713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154fb3b-6cf0-4f60-9240-76dd1e8fbe91 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:00.002899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:00.003721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:00.004145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:00.004434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] PATH_INFO: `/v3/projects/21b7f1c5eb6e428290770b219fc1f470/users/d3708a8d8e49444982e7da5b6eae634e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:00.026181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b7f1c5eb6e428290770b219fc1f470, user_id=d3708a8d8e49444982e7da5b6eae634e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:00.029230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:00.032041 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:00.053251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1321/2639] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:06:59 2022] PUT /identity/v3/projects/21b7f1c5eb6e428290770b219fc1f470/users/d3708a8d8e49444982e7da5b6eae634e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:00.057346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:00.070997 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2713c-94c3-43bb-9b65-74563a17b921 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:00.125596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:00.167940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8aff4-fd91-435d-af9a-48b907d0cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:00.168567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8aff4-fd91-435d-af9a-48b907d0cd4a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:00.168854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8aff4-fd91-435d-af9a-48b907d0cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:00.169131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8aff4-fd91-435d-af9a-48b907d0cd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:00.169795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8aff4-fd91-435d-af9a-48b907d0cd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:00.171980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8aff4-fd91-435d-af9a-48b907d0cd4a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:00.184803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1319/2640] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:00.189613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:00.201318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552e589-7818-4a05-9d3f-292c9c95d492 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:00.206810 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:00.207566 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:00.207906 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:00.208245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] PATH_INFO: `/v3/projects/21b7f1c5eb6e428290770b219fc1f470/users/d3708a8d8e49444982e7da5b6eae634e/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:00.210299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b7f1c5eb6e428290770b219fc1f470, user_id=d3708a8d8e49444982e7da5b6eae634e, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:00.213217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:00.227577 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1322/2641] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:00 2022] PUT /identity/v3/projects/21b7f1c5eb6e428290770b219fc1f470/users/d3708a8d8e49444982e7da5b6eae634e/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:07:00.238456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:00.238847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:00.239210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:00.274267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] MFA Rules not processed for user `d3708a8d8e49444982e7da5b6eae634e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:00.276499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:00.502833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1320/2642] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:00.513637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:00.515507 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:00.582091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34f442-d470-4f54-821e-8990d1cb34f2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:00.585671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:00.586311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:00.586592 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:00.586868 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:00.596362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:00.597864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:00.608453 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1323/2643] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:00.774787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:00.787672 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68603-1f49-4de7-938b-68c39ccd6ed0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:00.791316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:00.791953 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:00.792231 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:00.792500 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:00.873266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:00.875458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:00.888239 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1321/2644] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:03.148744 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:03.151079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:03.222241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374349a-1904-471f-a46b-529608c5c2f9 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:03.227030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:03.227892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.228345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.228858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] PATH_INFO: `/v3/users/cdbc898ffd4f4d5d87c3684fcc17313d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.230339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbc898ffd4f4d5d87c3684fcc17313d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:03.232317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:03.284209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbc898ffd4f4d5d87c3684fcc17313d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.291652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbc898ffd4f4d5d87c3684fcc17313d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.304899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbc898ffd4f4d5d87c3684fcc17313d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.305604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbc898ffd4f4d5d87c3684fcc17313d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.313227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7ba03746-030e-46e3-9e10-2033c82233b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbc898ffd4f4d5d87c3684fcc17313d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.319674 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1324/2645] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:03 2022] DELETE /identity/v3/users/cdbc898ffd4f4d5d87c3684fcc17313d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:03.499482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:03.510390 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf08d8-8c80-487c-9e79-96861f07776f service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:03.563912 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:03.571940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:03.572606 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.572887 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.573161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] PATH_INFO: `/v3/projects/ea560f6e50034738b03e588ada15e940` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.574313 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea560f6e50034738b03e588ada15e940)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:03.576210 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:03.632103 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea560f6e50034738b03e588ada15e940', 'request_id': 'req-8a11b6cf-30eb-40c4-b381-afca6159d07c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.638463 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea560f6e50034738b03e588ada15e940', 'request_id': 'req-8a11b6cf-30eb-40c4-b381-afca6159d07c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.645186 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea560f6e50034738b03e588ada15e940', 'request_id': 'req-8a11b6cf-30eb-40c4-b381-afca6159d07c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.659461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] The token cache is being invalidate because project ea560f6e50034738b03e588ada15e940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:03.659762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.660412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8a11b6cf-30eb-40c4-b381-afca6159d07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:03.662010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1322/2646] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:03 2022] DELETE /identity/v3/projects/ea560f6e50034738b03e588ada15e940 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:03.666850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.667135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.667409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.675503 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:03.689410 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:03.690651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:03.752944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1325/2647] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:03.761085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.761372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.761648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.782406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:03.783614 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:03.791491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1323/2648] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:03.796620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:03.806586 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c7a65-1ca9-4a8c-a1e6-5dcbf0bc0695 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:03.811222 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:03.811836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.812118 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.812392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.813065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:03.813373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:03.815035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:03.826833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1326/2649] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:03.831479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:03.841755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0a2c4-d71c-4152-b417-0c2561412721 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:03.846979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:03.847582 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.847861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.848134 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.848775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:03.850692 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:03.878987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1324/2650] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:03 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:07:03.883127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:03.894395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e30b8-f229-4dd8-abfb-a4e6ee5f8268 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:03.950568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:03.951208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.952024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.952302 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.952955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:03.954918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:03.976759 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1327/2651] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:03 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:03.981006 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:03.992502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0c0d7-9c24-4302-b6dc-716b631d55f9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:03.997200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:03.997804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:03.998120 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:03.998394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:03.999059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:04.000636 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:04.011072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1325/2652] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:04.015961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:04.026590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bb561-dbf1-446d-83df-67398cc550e8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:04.031904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0a705-62a3-4bf8-ae34-a7c76cb25d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:04.032518 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0a705-62a3-4bf8-ae34-a7c76cb25d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:04.032795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0a705-62a3-4bf8-ae34-a7c76cb25d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:04.033063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0a705-62a3-4bf8-ae34-a7c76cb25d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:04.033785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0a705-62a3-4bf8-ae34-a7c76cb25d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:04.035554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0a705-62a3-4bf8-ae34-a7c76cb25d2b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:04.048080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1328/2653] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:04.052383 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:04.063409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c047167-4877-4db3-87a3-d1ada4b75406 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:04.068037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:04.068687 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:04.068966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:04.069237 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] PATH_INFO: `/v3/projects/a860b6979be5483dba888a37634d196b/users/5d10a2be3765418cb72eeb6b15db5142/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:04.084129 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a860b6979be5483dba888a37634d196b, user_id=5d10a2be3765418cb72eeb6b15db5142, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:04.086813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:04.100240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1326/2654] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:04 2022] PUT /identity/v3/projects/a860b6979be5483dba888a37634d196b/users/5d10a2be3765418cb72eeb6b15db5142/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:04.105585 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80867eca-96c6-4dd8-92f3-513d3e1d9ef9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:04.105980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80867eca-96c6-4dd8-92f3-513d3e1d9ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:04.106198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80867eca-96c6-4dd8-92f3-513d3e1d9ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:04.127449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-80867eca-96c6-4dd8-92f3-513d3e1d9ef9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:04.128663 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-80867eca-96c6-4dd8-92f3-513d3e1d9ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:04.187869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1329/2655] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:04.197938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:04.199648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:04.257806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ccd1a-f419-4e3d-8209-45a741ce58b7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:04.261567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:04.262142 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:04.262427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:04.262702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:04.271624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:04.273057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:04.283284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1327/2656] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:08.614898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:08.615429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:08.615429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:08.637635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:08.647438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] MFA Rules not processed for user `5d10a2be3765418cb72eeb6b15db5142`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:08.648973 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:08.714992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1330/2657] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:08.729840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:08.731462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:08.770664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:08.828560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0bb71-021c-47cb-8482-fb7513fad789 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:08.836126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83ecd-9450-4c4a-a010-ed93d1b6606d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:08.837613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83ecd-9450-4c4a-a010-ed93d1b6606d service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:08.837894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83ecd-9450-4c4a-a010-ed93d1b6606d service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:08.838295 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83ecd-9450-4c4a-a010-ed93d1b6606d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:08.848198 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83ecd-9450-4c4a-a010-ed93d1b6606d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:08.849772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83ecd-9450-4c4a-a010-ed93d1b6606d service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:08.859209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1328/2658] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:09.802079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:09.803594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:09.822276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af51fe-644c-4ae3-bb44-fa0f41524a6d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:09.828240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:09.829033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:09.829322 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:09.829624 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] PATH_INFO: `/v3/users/18bceb4905ac4d799a93e60d3b684180` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:09.831611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bceb4905ac4d799a93e60d3b684180)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:09.834113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:09.890363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bceb4905ac4d799a93e60d3b684180'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:09.901862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bceb4905ac4d799a93e60d3b684180'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:09.922799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bceb4905ac4d799a93e60d3b684180'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:09.923726 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bceb4905ac4d799a93e60d3b684180'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:09.943745 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bceb4905ac4d799a93e60d3b684180'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:09.952382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1331/2659] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:09 2022] DELETE /identity/v3/users/18bceb4905ac4d799a93e60d3b684180 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:07:09.957027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b3ad1-7518-4512-939a-a9cc3ba663a5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:09.957536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b3ad1-7518-4512-939a-a9cc3ba663a5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:09.957646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b3ad1-7518-4512-939a-a9cc3ba663a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:09.979357 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-494b3ad1-7518-4512-939a-a9cc3ba663a5 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:09.980612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-494b3ad1-7518-4512-939a-a9cc3ba663a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:10.056335 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1329/2660] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:10.071819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:10.073587 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:10.158591 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a9492-36ef-4bb5-8bec-a7b2dd62d241 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:10.162483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:10.163296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:10.163748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:10.164183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:10.174378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:10.175843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:10.187005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1332/2661] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:10.319721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:10.320013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:10.320293 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:10.348341 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] MFA Rules not processed for user `5d10a2be3765418cb72eeb6b15db5142`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:10.351218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:10.420444 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1330/2662] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:10.478053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:10.479468 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:10.502776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ff471-844b-4bb6-808b-c088344a6b29 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:10.507779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:10.510593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:10.511047 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:10.511300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] PATH_INFO: `/v3/projects/7816f3fa6d424d48ac2dea3dda45b0e3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:10.512712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7816f3fa6d424d48ac2dea3dda45b0e3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:10.515771 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:10.524790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:10.569493 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7816f3fa6d424d48ac2dea3dda45b0e3', 'request_id': 'req-3bcc3773-a678-4d69-9b88-8c575a952374'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.575079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7816f3fa6d424d48ac2dea3dda45b0e3', 'request_id': 'req-3bcc3773-a678-4d69-9b88-8c575a952374'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.581803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7816f3fa6d424d48ac2dea3dda45b0e3', 'request_id': 'req-3bcc3773-a678-4d69-9b88-8c575a952374'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.595350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] The token cache is being invalidate because project 7816f3fa6d424d48ac2dea3dda45b0e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:10.595646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.596273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.598438 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1333/2663] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:10 2022] DELETE /identity/v3/projects/7816f3fa6d424d48ac2dea3dda45b0e3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:10.602468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:10.613570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063341e-fb98-44f7-abdf-1193e4effed9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:10.684178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:10.693428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:10.694112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:10.694392 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:10.694663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] PATH_INFO: `/v3/users/d3708a8d8e49444982e7da5b6eae634e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:10.695900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3708a8d8e49444982e7da5b6eae634e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:10.697823 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:10.746652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3708a8d8e49444982e7da5b6eae634e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.755200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3708a8d8e49444982e7da5b6eae634e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.767440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3708a8d8e49444982e7da5b6eae634e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.768208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3708a8d8e49444982e7da5b6eae634e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.776045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ad71eddf-5dbb-4bd8-b613-6203d4479df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3708a8d8e49444982e7da5b6eae634e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:10.782782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1331/2664] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:10 2022] DELETE /identity/v3/users/d3708a8d8e49444982e7da5b6eae634e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:11.105965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.111115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:11.146006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc3773-a678-4d69-9b88-8c575a952374 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.276296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.277292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.277785 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.278284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] PATH_INFO: `/v3/projects/21b7f1c5eb6e428290770b219fc1f470` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.279689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b7f1c5eb6e428290770b219fc1f470)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.281842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.348169 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b7f1c5eb6e428290770b219fc1f470', 'request_id': 'req-712973b7-2e82-4f3a-9282-5e799ac0e851'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:11.355064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b7f1c5eb6e428290770b219fc1f470', 'request_id': 'req-712973b7-2e82-4f3a-9282-5e799ac0e851'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:11.362328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b7f1c5eb6e428290770b219fc1f470', 'request_id': 'req-712973b7-2e82-4f3a-9282-5e799ac0e851'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:11.379927 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] The token cache is being invalidate because project 21b7f1c5eb6e428290770b219fc1f470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:11.380712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:11.381644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-712973b7-2e82-4f3a-9282-5e799ac0e851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:11.384025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1334/2665] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:11 2022] DELETE /identity/v3/projects/21b7f1c5eb6e428290770b219fc1f470 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:11.388718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.389784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.390065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.410996 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:11.412443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:11.490294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1332/2666] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:11.498979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.499270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.499558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.521563 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:11.522864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:11.531813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1335/2667] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:11.536017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.548580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a15e7-9dc9-4306-8a85-57faf34d2c93 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.554294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.554834 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.555344 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.557173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.557367 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:11.557665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.559560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.574462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1333/2668] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:11.579070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.589603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f905f02-f7c5-4bb7-93a5-743889d1da17 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.595705 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.596450 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.596786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.597070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.598299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.601158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.628173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:11.638445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1336/2669] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:11 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:11.642599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.653451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520511-ec92-410f-aeaa-454e831692d7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.717062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.717584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.717866 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.718238 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.718891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.720825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.730833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:11.746498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1334/2670] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:11 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:11.751353 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.763757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5bb9c-b5e8-45bf-b051-128bd84c64ba admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.768786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.769399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.769679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.769980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.770631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.772314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.783523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1337/2671] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:11.787141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.797369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6d76a-380e-4bd0-954d-3a3d7d15fe91 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.801845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd4ede-c070-40a3-8c2e-8bbd96f5edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.802612 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd4ede-c070-40a3-8c2e-8bbd96f5edbf admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.802747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd4ede-c070-40a3-8c2e-8bbd96f5edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.802990 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd4ede-c070-40a3-8c2e-8bbd96f5edbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.803621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd4ede-c070-40a3-8c2e-8bbd96f5edbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.805201 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd4ede-c070-40a3-8c2e-8bbd96f5edbf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.815817 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1335/2672] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:11.820150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:11.831105 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591d05-a3b7-4015-bb27-87a9ca8f1f61 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:11.837038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:11.837967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.838121 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.838482 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] PATH_INFO: `/v3/projects/8865f82024d44cca803e0fb2dd34bb45/users/18a760bec68a4969a264e11d759c1127/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.855860 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8865f82024d44cca803e0fb2dd34bb45, user_id=18a760bec68a4969a264e11d759c1127, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:11.859643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:11.885243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1338/2673] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:11 2022] PUT /identity/v3/projects/8865f82024d44cca803e0fb2dd34bb45/users/18a760bec68a4969a264e11d759c1127/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:11.887326 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a974fa-7173-4831-8831-51fa7d508a11 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:11.887624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a974fa-7173-4831-8831-51fa7d508a11 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:11.887881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a974fa-7173-4831-8831-51fa7d508a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:11.917359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d6a974fa-7173-4831-8831-51fa7d508a11 None None] MFA Rules not processed for user `18a760bec68a4969a264e11d759c1127`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:11.919022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6a974fa-7173-4831-8831-51fa7d508a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:12.009296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1336/2674] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:12.021168 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:12.022889 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:12.083126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda6df-14df-4c93-a3dc-02748aa25b61 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:12.087245 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:12.087853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:12.088251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:12.088538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:12.097731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:12.099367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:12.109127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1339/2675] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:14.447204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:14.447756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:14.447756 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:14.466683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] MFA Rules not processed for user `5a93a2d9266541e7bff502ba074fef5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:14.467970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:14.534159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1337/2676] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:14.884512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:14.885919 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:14.941621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a51b8-7dd0-4594-beca-2a60d30e5dcd service glance] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:14.945261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:14.945866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:14.946253 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:14.946532 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:14.948393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:14.957883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:14.959472 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23368af3-805e-4337-a575-300b20584d22 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:14.967863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1340/2677] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:24.527521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=2,keystone:SELECT=64 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:24.963788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:29.909551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:29.973371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f56d-2ae4-40ad-ad5d-b03ae31b565b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:29.977880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:29.978542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:29.978825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:29.979101 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] PATH_INFO: `/v3/users/18a760bec68a4969a264e11d759c1127` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:29.980214 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a760bec68a4969a264e11d759c1127)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:29.982719 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.026898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a760bec68a4969a264e11d759c1127'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.034297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a760bec68a4969a264e11d759c1127'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.046704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a760bec68a4969a264e11d759c1127'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.047334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a760bec68a4969a264e11d759c1127'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.054380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a760bec68a4969a264e11d759c1127'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.060128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1338/2678] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:29 2022] DELETE /identity/v3/users/18a760bec68a4969a264e11d759c1127 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:30.064598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.065106 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.065207 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.084080 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:30.085272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:30.143961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1341/2679] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:30.153523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.154981 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:30.212926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f120e3-1a94-4cd4-ab15-e80698b9302a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.216556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53626d8-0e8f-4d31-9c64-d240fc5c9621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.217155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53626d8-0e8f-4d31-9c64-d240fc5c9621 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.217434 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53626d8-0e8f-4d31-9c64-d240fc5c9621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.217707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53626d8-0e8f-4d31-9c64-d240fc5c9621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.226470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53626d8-0e8f-4d31-9c64-d240fc5c9621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.227884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53626d8-0e8f-4d31-9c64-d240fc5c9621 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.237620 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1339/2680] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:30.524390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.525841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:30.538498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2691d1-fb81-40a3-961d-d1fc0a5ac15e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.543246 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.543969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.544317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.544730 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] PATH_INFO: `/v3/projects/8865f82024d44cca803e0fb2dd34bb45` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.546976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8865f82024d44cca803e0fb2dd34bb45)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.549477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.595762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8865f82024d44cca803e0fb2dd34bb45', 'request_id': 'req-de70c4ea-14d0-4484-8f5e-b3c409d4c351'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.601360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8865f82024d44cca803e0fb2dd34bb45', 'request_id': 'req-de70c4ea-14d0-4484-8f5e-b3c409d4c351'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.608165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8865f82024d44cca803e0fb2dd34bb45', 'request_id': 'req-de70c4ea-14d0-4484-8f5e-b3c409d4c351'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.621311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] The token cache is being invalidate because project 8865f82024d44cca803e0fb2dd34bb45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:30.621603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.622447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de70c4ea-14d0-4484-8f5e-b3c409d4c351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:30.623937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1342/2681] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:30 2022] DELETE /identity/v3/projects/8865f82024d44cca803e0fb2dd34bb45 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:30.628851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.629137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.629410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.648283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:30.649461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:30.667042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:30.710160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1340/2682] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:30.718569 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.718853 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.719122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.741009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:30.742644 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:30.750740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1343/2683] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:30.755275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.765187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b103edf-c155-4ccd-88f8-36f7c4dcbdab None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.769978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.770600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.770878 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.771153 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.771793 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:30.772095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.773801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.785596 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1341/2684] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:30.789804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.799693 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bad8e0-dc4f-4c66-aba5-37677def610c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.804495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.805094 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.805368 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.805661 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.806334 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.808224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.834059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1344/2685] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:30.838255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.848746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40375e5b-41d5-4af2-9e2a-59dd4458529e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.906712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.907312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.907597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.907874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.908504 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.910450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.931426 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1342/2686] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:30 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:30.935671 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.945807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3dd3-8d9a-4166-a18d-0426c79c1745 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.950772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.951790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.952073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.952346 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.952987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.954658 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:30.965492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1345/2687] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:30.970160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:30.980339 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404b02d-384e-4cae-b2ed-d46b4911caaa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:30.985046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb109fd8-1526-4e44-b82a-703c311832d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:30.985642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109fd8-1526-4e44-b82a-703c311832d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:30.985949 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109fd8-1526-4e44-b82a-703c311832d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:30.986232 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109fd8-1526-4e44-b82a-703c311832d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:30.986889 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb109fd8-1526-4e44-b82a-703c311832d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:30.988453 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb109fd8-1526-4e44-b82a-703c311832d4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:31.010155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1343/2688] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:31.011464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:31.022311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331782-ddec-4ac5-a632-c7ff33bc8cd0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:31.027059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:31.027729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:31.027989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:31.028255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] PATH_INFO: `/v3/projects/1683e254a7bd46c2bd3607b87e01dad6/users/339ec702e5ef416b8a9448d6f8ab8e9a/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:31.044031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1683e254a7bd46c2bd3607b87e01dad6, user_id=339ec702e5ef416b8a9448d6f8ab8e9a, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:31.046802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:31.061056 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1346/2689] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:31 2022] PUT /identity/v3/projects/1683e254a7bd46c2bd3607b87e01dad6/users/339ec702e5ef416b8a9448d6f8ab8e9a/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:31.072385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:31.072683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:31.072963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:31.102250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] MFA Rules not processed for user `339ec702e5ef416b8a9448d6f8ab8e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:31.103399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:31.169000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1344/2690] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:31.179736 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:31.181617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:31.204240 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:31.241682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e669-c21f-4605-a092-6a7d75b1eba5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:31.245782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:31.246424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:31.246707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:31.246982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:31.257489 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:31.258969 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:31.268276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1347/2691] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:31.338683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:31.340069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:31.362621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:31.401402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825fdd8-8fdc-48fa-9992-e170d0cac745 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:31.405562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:31.406197 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:31.406486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:31.406765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:31.408630 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:31.480539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:31.482855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e49c42-7449-46bd-917a-29959f2313a0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:31.493189 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1345/2692] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:38.670258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:38.733446 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcfd5e-a211-4320-b4c1-2042f4482370 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:38.738823 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:38.739459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:38.739743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:38.740018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] PATH_INFO: `/v3/users/339ec702e5ef416b8a9448d6f8ab8e9a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:38.741135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=339ec702e5ef416b8a9448d6f8ab8e9a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:38.743395 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:38.788808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339ec702e5ef416b8a9448d6f8ab8e9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:38.796792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339ec702e5ef416b8a9448d6f8ab8e9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:38.808558 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339ec702e5ef416b8a9448d6f8ab8e9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:38.809201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339ec702e5ef416b8a9448d6f8ab8e9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:38.815944 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339ec702e5ef416b8a9448d6f8ab8e9a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:38.821680 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1348/2693] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:38 2022] DELETE /identity/v3/users/339ec702e5ef416b8a9448d6f8ab8e9a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:38.826191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:38.826666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:38.826772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:38.846789 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:38.847980 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:38.904277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1346/2694] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:38.914378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:38.915775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:38.965702 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:38.984514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b325be2-cde4-4b67-8e2d-a92a31b418ef admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:38.988198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256169b-3e71-4c52-8586-cd3bdd8d4c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:38.988808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256169b-3e71-4c52-8586-cd3bdd8d4c48 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:38.989091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256169b-3e71-4c52-8586-cd3bdd8d4c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:38.989364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256169b-3e71-4c52-8586-cd3bdd8d4c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:38.998682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256169b-3e71-4c52-8586-cd3bdd8d4c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.000113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256169b-3e71-4c52-8586-cd3bdd8d4c48 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.009478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1349/2695] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:39.263821 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.265836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:39.277975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ab95a-d06f-4a58-b731-417673a632ca None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.283484 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.284154 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.284468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.284707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] PATH_INFO: `/v3/projects/1683e254a7bd46c2bd3607b87e01dad6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.286043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1683e254a7bd46c2bd3607b87e01dad6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.288164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.342680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1683e254a7bd46c2bd3607b87e01dad6', 'request_id': 'req-1ac0fb4e-2406-4157-a94f-d49f8905eb44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:39.351156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1683e254a7bd46c2bd3607b87e01dad6', 'request_id': 'req-1ac0fb4e-2406-4157-a94f-d49f8905eb44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:39.357684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1683e254a7bd46c2bd3607b87e01dad6', 'request_id': 'req-1ac0fb4e-2406-4157-a94f-d49f8905eb44'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:39.359356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:39.373439 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] The token cache is being invalidate because project 1683e254a7bd46c2bd3607b87e01dad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:39.373725 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:39.374393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1ac0fb4e-2406-4157-a94f-d49f8905eb44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:39.375892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1347/2696] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:39 2022] DELETE /identity/v3/projects/1683e254a7bd46c2bd3607b87e01dad6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:39.381719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.382037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.382316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.402965 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:39.404196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:39.467696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1350/2697] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:39.470450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.470741 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.471016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.490243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:39.491435 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:39.499909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1348/2698] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:07:39.503936 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.513289 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39270-dc60-4cb2-969a-f72b4a0f7239 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.518393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.518982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.519259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.519529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.520192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:39.520494 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.522107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.533300 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1351/2699] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:39.537322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.546532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b39eb-263c-4717-a5e3-22d18b2b4784 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.551356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.551947 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.552230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.552505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.553140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.555113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.582624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1349/2700] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:39 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:07:39.586804 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.596581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf950e60-f2e0-4ba8-860b-c2964289515c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.652389 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.652998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.653282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.653615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.654272 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.660351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.684806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1352/2701] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:39 2022] POST /identity/v3/users => generated 280 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:39.689586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.700137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc05ae-4bf9-4914-aa4b-6e356c1641e4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.705894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.706662 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.706938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.707219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.707864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.709476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.719428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1350/2702] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:39.723201 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.733083 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a0bf5-172c-4880-85a2-ae42d6862d62 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.744146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222171f-f2e8-494c-a4a9-65d440578fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.744743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222171f-f2e8-494c-a4a9-65d440578fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.745023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222171f-f2e8-494c-a4a9-65d440578fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.745298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222171f-f2e8-494c-a4a9-65d440578fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.745977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222171f-f2e8-494c-a4a9-65d440578fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.747610 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222171f-f2e8-494c-a4a9-65d440578fdd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.759486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1353/2703] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:39.763584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.774012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fbaf0-2886-47c8-aa17-b7bd1c8adfb0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:39.778486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:39.779118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.779396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.779668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] PATH_INFO: `/v3/projects/2bd7e419fd924222b4d5942eb241d0a6/users/d6a53ea8935f475eaa814feef6203fca/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.793505 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd7e419fd924222b4d5942eb241d0a6, user_id=d6a53ea8935f475eaa814feef6203fca, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:39.796195 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:39.808345 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1351/2704] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:39 2022] PUT /identity/v3/projects/2bd7e419fd924222b4d5942eb241d0a6/users/d6a53ea8935f475eaa814feef6203fca/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:39.818978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:39.819270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:39.819548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:39.846838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] MFA Rules not processed for user `d6a53ea8935f475eaa814feef6203fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:39.848034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:39.888945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:39.912911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1354/2705] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:39.947744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:39.949445 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:40.017646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17fbb-6491-453c-97dd-5fa6d8e69bdb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:40.021173 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559858-b607-492d-93b0-431a0d8fd166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:40.021774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559858-b607-492d-93b0-431a0d8fd166 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.022094 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559858-b607-492d-93b0-431a0d8fd166 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.022372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559858-b607-492d-93b0-431a0d8fd166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.031027 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559858-b607-492d-93b0-431a0d8fd166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:40.032452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559858-b607-492d-93b0-431a0d8fd166 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:40.041084 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1352/2706] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:39 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:40.056393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:40.117473 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7cce4-444b-41ea-9046-0506aa045ec8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:40.122123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:40.122742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.123023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.123299 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] PATH_INFO: `/v3/users/d6a53ea8935f475eaa814feef6203fca` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.124429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a53ea8935f475eaa814feef6203fca)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:40.126393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:40.171464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a53ea8935f475eaa814feef6203fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.179365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a53ea8935f475eaa814feef6203fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.191333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a53ea8935f475eaa814feef6203fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.191971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a53ea8935f475eaa814feef6203fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.198718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a53ea8935f475eaa814feef6203fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.204952 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1355/2707] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:40 2022] DELETE /identity/v3/users/d6a53ea8935f475eaa814feef6203fca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:40.209465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.209747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.210049 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.228894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:40.230102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:40.289056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1353/2708] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:40.299354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:40.300991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:40.357571 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc65c4-f18b-4167-942e-4ab2f853e1bd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:40.361183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65ed4d-7a9b-41d7-8468-c24b46ccc5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:40.361782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65ed4d-7a9b-41d7-8468-c24b46ccc5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.362104 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65ed4d-7a9b-41d7-8468-c24b46ccc5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.362382 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65ed4d-7a9b-41d7-8468-c24b46ccc5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.371405 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65ed4d-7a9b-41d7-8468-c24b46ccc5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:40.372824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65ed4d-7a9b-41d7-8468-c24b46ccc5d5 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:40.382716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1356/2709] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:40.648758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:40.650233 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:40.662695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da541906-d0f6-43e1-bf05-5c6a32bf1d3b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:40.667249 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:40.667876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.668159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.668429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] PATH_INFO: `/v3/projects/2bd7e419fd924222b4d5942eb241d0a6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.669539 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd7e419fd924222b4d5942eb241d0a6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:40.672108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:40.703174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:40.720462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd7e419fd924222b4d5942eb241d0a6', 'request_id': 'req-68b6ad3c-788f-4876-95d2-30746bb3c121'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.725979 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd7e419fd924222b4d5942eb241d0a6', 'request_id': 'req-68b6ad3c-788f-4876-95d2-30746bb3c121'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.732053 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd7e419fd924222b4d5942eb241d0a6', 'request_id': 'req-68b6ad3c-788f-4876-95d2-30746bb3c121'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.744608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] The token cache is being invalidate because project 2bd7e419fd924222b4d5942eb241d0a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:40.744899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.745532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-68b6ad3c-788f-4876-95d2-30746bb3c121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:40.747029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1354/2710] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:40 2022] DELETE /identity/v3/projects/2bd7e419fd924222b4d5942eb241d0a6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:40.752995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.753471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.753683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.761635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:40.775974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:40.777152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:40.842895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1357/2711] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:40.851720 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.852010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.852292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.872359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:40.873557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:40.882002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1355/2712] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:40.886328 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:40.895915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c855c50-4628-443d-9e6c-8a6aa4218e76 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:40.900445 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:40.901036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.901317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.901590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.902267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:40.902572 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:40.904162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:40.915416 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1358/2713] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:40.919682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:40.929615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ab402-3522-47a0-8300-2b0cd0a4f4e6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:40.936174 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:40.937368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:40.937652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:40.937950 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:40.938592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:40.940480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:40.967114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1356/2714] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:40 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:40.971790 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:40.981811 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a639b-c51c-4246-b34b-a0605756d820 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:41.036954 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:41.037583 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:41.037866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:41.038238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:41.038896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:41.040824 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:41.062142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1359/2715] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:40 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:41.066194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:41.076635 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bff2-3489-4fce-b758-615b66fbc6a9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:41.081222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:41.081812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:41.082131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:41.082407 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:41.083044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:41.084650 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:41.095082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1357/2716] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:41.098773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:41.109335 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cb1a7-b1cd-47d2-9456-aa79adbdc448 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:41.113937 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17c8d2-3b3b-4007-8b02-889a025473d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:41.114525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17c8d2-3b3b-4007-8b02-889a025473d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:41.114809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17c8d2-3b3b-4007-8b02-889a025473d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:41.115085 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17c8d2-3b3b-4007-8b02-889a025473d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:41.115721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17c8d2-3b3b-4007-8b02-889a025473d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:41.117570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17c8d2-3b3b-4007-8b02-889a025473d7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:41.127938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1360/2717] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:41.131841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:41.141513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c177c-5ab5-4e5a-94cf-8ba79618856b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:41.146048 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:41.146674 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:41.146954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:41.147226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] PATH_INFO: `/v3/projects/c25819e17b344d32a1a93c924070c84a/users/f62cf3c87097467da462f3420a24971d/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:41.162702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25819e17b344d32a1a93c924070c84a, user_id=f62cf3c87097467da462f3420a24971d, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:41.165296 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:41.178178 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1358/2718] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:41 2022] PUT /identity/v3/projects/c25819e17b344d32a1a93c924070c84a/users/f62cf3c87097467da462f3420a24971d/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:07:41.188444 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:41.188735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:41.189006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:41.215750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] MFA Rules not processed for user `f62cf3c87097467da462f3420a24971d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:41.216951 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:41.274995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:41.281689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1361/2719] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:41.292259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:41.294010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:41.350338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78d038-8254-45d8-8758-d1173396de71 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:41.353844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd00989-aaa5-4d9b-8740-191e6de230f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:41.354493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd00989-aaa5-4d9b-8740-191e6de230f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:41.354773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd00989-aaa5-4d9b-8740-191e6de230f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:41.355045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd00989-aaa5-4d9b-8740-191e6de230f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:41.363155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd00989-aaa5-4d9b-8740-191e6de230f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:41.364563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd00989-aaa5-4d9b-8740-191e6de230f2 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:41.373333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1359/2720] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:44.381360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:44.441311 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16f833-1eda-4173-a30c-e3313cdee790 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:44.445792 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:44.446430 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:44.446708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:44.446978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] PATH_INFO: `/v3/users/f62cf3c87097467da462f3420a24971d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:44.448064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62cf3c87097467da462f3420a24971d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:44.450109 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:44.495686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62cf3c87097467da462f3420a24971d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:44.502573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62cf3c87097467da462f3420a24971d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:44.514064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62cf3c87097467da462f3420a24971d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:44.515194 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62cf3c87097467da462f3420a24971d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:44.522013 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62cf3c87097467da462f3420a24971d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:44.527757 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1362/2721] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:44 2022] DELETE /identity/v3/users/f62cf3c87097467da462f3420a24971d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:44.532835 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:44.533305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:44.533418 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:44.552031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:44.553155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:44.599283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:44.612669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1360/2722] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:44.622645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:44.624071 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:44.681141 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1445f1-e22f-4ca4-b27a-0936060a253e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:44.684835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73361e73-a148-4ee9-bea3-76078353a5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:44.685466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73361e73-a148-4ee9-bea3-76078353a5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:44.685750 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73361e73-a148-4ee9-bea3-76078353a5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:44.686060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73361e73-a148-4ee9-bea3-76078353a5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:44.694918 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73361e73-a148-4ee9-bea3-76078353a5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:44.696354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73361e73-a148-4ee9-bea3-76078353a5a3 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:44.706225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1363/2723] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:44.966080 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:44.967502 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:44.979599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e2bb7-9a32-4a7f-be5a-23d73f76e34d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:44.984342 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:44.984973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:44.985254 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:44.985558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] PATH_INFO: `/v3/projects/c25819e17b344d32a1a93c924070c84a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:44.986745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25819e17b344d32a1a93c924070c84a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:44.988654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.042758 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25819e17b344d32a1a93c924070c84a', 'request_id': 'req-30398364-3f3e-464a-adcd-504126e4c46b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:45.048122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25819e17b344d32a1a93c924070c84a', 'request_id': 'req-30398364-3f3e-464a-adcd-504126e4c46b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:45.054220 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25819e17b344d32a1a93c924070c84a', 'request_id': 'req-30398364-3f3e-464a-adcd-504126e4c46b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:45.067446 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] The token cache is being invalidate because project c25819e17b344d32a1a93c924070c84a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:45.067765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:45.068386 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30398364-3f3e-464a-adcd-504126e4c46b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:45.069894 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1361/2724] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:44 2022] DELETE /identity/v3/projects/c25819e17b344d32a1a93c924070c84a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:45.075022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.075337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.075648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.215442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:45.216652 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:45.230904 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:45.283173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1364/2725] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:45.291070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.291355 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.291627 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.312703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:45.313948 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:45.322158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1362/2726] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:45.326723 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.337249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23560-c4e2-4a19-8045-f56a1568ee64 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.342616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.343251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.343531 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.343802 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.344501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:45.344806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.346424 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.363812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1365/2727] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:45.370429 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.384113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1284a-b73b-4884-8311-21d4195bb0cc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.388968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.389573 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.389852 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.390148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.390818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.392706 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.420616 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1363/2728] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:07:45.425305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.435529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efa05e-41f2-44a9-8100-d8881c594eb6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.491940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.492795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.493073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.493343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.493984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.495880 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.516755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1366/2729] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:45.522327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.533095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23459d-36f3-4130-8822-b637f9ea3d92 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.537655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.538314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.538600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.538879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.539519 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.541100 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.551544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1364/2730] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:45.555619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.568386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a85b66-fd4f-45e4-a9aa-c3e42b696219 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.573143 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fd95-43e4-4bab-a476-d6d83af3023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.573733 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fd95-43e4-4bab-a476-d6d83af3023a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.574053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fd95-43e4-4bab-a476-d6d83af3023a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.574323 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fd95-43e4-4bab-a476-d6d83af3023a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.575070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fd95-43e4-4bab-a476-d6d83af3023a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.576646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fd95-43e4-4bab-a476-d6d83af3023a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.586908 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1367/2731] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:45.592322 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.602913 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69269fa4-3589-4a73-913c-84507ec3c9b2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.607428 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.608073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.608939 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.609230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] PATH_INFO: `/v3/projects/942a34d851b04723b680b90575ad70fa/users/3a2f26b97ba84639a0fa1ab1f7e0e903/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.628286 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=942a34d851b04723b680b90575ad70fa, user_id=3a2f26b97ba84639a0fa1ab1f7e0e903, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.631052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.646106 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1365/2732] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:45 2022] PUT /identity/v3/projects/942a34d851b04723b680b90575ad70fa/users/3a2f26b97ba84639a0fa1ab1f7e0e903/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:45.712975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.713254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.713526 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.741916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] MFA Rules not processed for user `3a2f26b97ba84639a0fa1ab1f7e0e903`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:45.747075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:45.808912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1368/2733] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:45.814698 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.878639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff726c8-212a-4238-8ccf-b508bb3dfbce admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.883316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.883922 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.884207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.884482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.885112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.887045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:45.908556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:45.916389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1366/2734] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:07:45.920659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:45.931441 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83609f11-e6d5-4238-b8ae-8714a6c41174 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:45.942955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:45.988463 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:45.989070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:45.989351 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:45.989620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:45.990265 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:45.992433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.013990 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1369/2735] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:45 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:46.018490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.028801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef148e-3441-40c4-8700-c39670405881 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.034222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.034816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.035098 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.035375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.036011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.037597 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.047934 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1367/2736] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:46.051862 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.061654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29724ab-afb3-48e9-b814-920b72d1460f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.066284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3d2eb-65e7-4a68-beeb-83780aa1b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.066871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3d2eb-65e7-4a68-beeb-83780aa1b034 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.067149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3d2eb-65e7-4a68-beeb-83780aa1b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.067420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3d2eb-65e7-4a68-beeb-83780aa1b034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.068081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3d2eb-65e7-4a68-beeb-83780aa1b034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.070203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3d2eb-65e7-4a68-beeb-83780aa1b034 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.080916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1370/2737] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:46.085050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.095520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e830f-4600-438e-ba44-0aed794fa42e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.100051 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.100689 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.100971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.101246 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] PATH_INFO: `/v3/projects/8b4c36c25d7e4865bde6dc4c239278c2/users/93b04d7d46164befaf5595fe0093f516/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.115387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4c36c25d7e4865bde6dc4c239278c2, user_id=93b04d7d46164befaf5595fe0093f516, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.118343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.131184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1368/2738] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:46 2022] PUT /identity/v3/projects/8b4c36c25d7e4865bde6dc4c239278c2/users/93b04d7d46164befaf5595fe0093f516/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:46.141835 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.142149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.142421 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.169606 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] MFA Rules not processed for user `93b04d7d46164befaf5595fe0093f516`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:46.170962 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:46.232903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1371/2739] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:46.237340 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.299580 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2722cc-cd29-44b3-8503-9d13ddd33fcc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.304121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.304713 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.304994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.305265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.305884 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.307796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.335118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1369/2740] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:46 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:46.339942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.350688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f71d9-d7e2-4d5f-8a9d-b7ddc315a279 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.407448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.408051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.408332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.408605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.409234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.411186 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.431498 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1372/2741] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:46 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:46.435622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.446410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762eec7-ca38-4945-b122-22b1a49c240f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.451030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.451621 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.451906 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.452854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.453524 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.455150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.466474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1370/2742] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:46.470539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.480347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d03ac2-d02e-4fd5-aa71-d20a3818fe98 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.485005 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.485641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.485950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.486230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] PATH_INFO: `/v3/projects/08bedbaf52ad4b9699683ba2d1b03016/users/2f69e110d4e643b185ba8819677dec74/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.497383 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:46.502565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bedbaf52ad4b9699683ba2d1b03016, user_id=2f69e110d4e643b185ba8819677dec74, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.505758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.518822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1373/2743] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:46 2022] PUT /identity/v3/projects/08bedbaf52ad4b9699683ba2d1b03016/users/2f69e110d4e643b185ba8819677dec74/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:07:46.522683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.532328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb253c41-08c2-4e75-a42d-b31b232c899d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.587196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b581e-fd1c-4daf-a472-58799236c8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.587800 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b581e-fd1c-4daf-a472-58799236c8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.587991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b581e-fd1c-4daf-a472-58799236c8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.588257 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b581e-fd1c-4daf-a472-58799236c8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.588882 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b581e-fd1c-4daf-a472-58799236c8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.590477 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b581e-fd1c-4daf-a472-58799236c8ac admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.599988 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1371/2744] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:46.604176 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.614781 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6e9f6-e862-483b-92cb-f4896f8b789e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.619627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.620268 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.620551 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.620825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] PATH_INFO: `/v3/projects/08bedbaf52ad4b9699683ba2d1b03016/users/2f69e110d4e643b185ba8819677dec74/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.622755 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bedbaf52ad4b9699683ba2d1b03016, user_id=2f69e110d4e643b185ba8819677dec74, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.626069 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.636554 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1374/2745] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:46 2022] PUT /identity/v3/projects/08bedbaf52ad4b9699683ba2d1b03016/users/2f69e110d4e643b185ba8819677dec74/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:46.647718 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.648000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.648272 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.676250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] MFA Rules not processed for user `2f69e110d4e643b185ba8819677dec74`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:46.677465 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:46.697805 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:46.745452 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1372/2746] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:46.757884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:46.759698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:46.817171 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870233a-73fd-4cc8-98de-7da62e0eb0e1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:46.820743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:46.821392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:46.821667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:46.821966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:46.877185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:46.878639 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:46.887284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1375/2747] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:48.384207 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:48.385677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:48.397436 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b5400-e506-4688-86ad-61e8e2a10217 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:48.400896 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:48.401482 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:48.401760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:48.402069 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:48.456065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:48.457475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:48.466011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1373/2748] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:48.981340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:48.990425 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f54ea-a859-449c-b034-d15d1841488c service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:48.994449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:48.995051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:48.995332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:48.995630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:49.004989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:49.006447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:49.015550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1376/2749] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:52.912060 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:52.976663 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9d6d9-cd6f-42a7-b6fe-2b8561d7c376 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:52.981219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:52.981845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:52.982162 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:52.982456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] PATH_INFO: `/v3/users/3a2f26b97ba84639a0fa1ab1f7e0e903` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:52.984151 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2f26b97ba84639a0fa1ab1f7e0e903)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:52.986089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:53.031508 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2f26b97ba84639a0fa1ab1f7e0e903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.039283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2f26b97ba84639a0fa1ab1f7e0e903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.040355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:53.041871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:53.045102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:53.056565 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2f26b97ba84639a0fa1ab1f7e0e903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.057352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2f26b97ba84639a0fa1ab1f7e0e903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.064982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f641ea86-cce9-4c8a-87e3-e83869ea92e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2f26b97ba84639a0fa1ab1f7e0e903'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.074970 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1374/2750] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:52 2022] DELETE /identity/v3/users/3a2f26b97ba84639a0fa1ab1f7e0e903 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:53.076356 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:53.076634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:53.076938 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:53.100347 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:53.101560 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:53.107945 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d864-a4ea-460b-8c69-ffea68c55e52 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:53.143500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:53.165665 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1375/2751] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:53.172030 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:53.172637 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:53.172916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:53.173185 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:53.175174 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:53.176252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:53.177757 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:53.191488 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab239d-de71-47ce-847c-0321930ce682 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:53.195125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:53.195730 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:53.196013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:53.196287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:53.205873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:53.207400 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:53.216797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1376/2752] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:07:53.237049 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:53.238530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:53.247565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1377/2753] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:07:53.494205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:53.495832 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:53.513324 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71ff3-c7bb-4181-9bf2-73c1ffd11104 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:53.518862 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:53.519420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:53.519708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:53.519986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] PATH_INFO: `/v3/projects/942a34d851b04723b680b90575ad70fa` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:53.521164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942a34d851b04723b680b90575ad70fa)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:53.523042 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:53.582283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942a34d851b04723b680b90575ad70fa', 'request_id': 'req-24cea6df-931c-4798-bd10-ee073bc55408'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.588954 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942a34d851b04723b680b90575ad70fa', 'request_id': 'req-24cea6df-931c-4798-bd10-ee073bc55408'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.597362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942a34d851b04723b680b90575ad70fa', 'request_id': 'req-24cea6df-931c-4798-bd10-ee073bc55408'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.610550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] The token cache is being invalidate because project 942a34d851b04723b680b90575ad70fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:53.610850 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.611493 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.612985 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1377/2754] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:53 2022] DELETE /identity/v3/projects/942a34d851b04723b680b90575ad70fa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:53.617461 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:53.627794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c75b2-378f-49de-a88e-e1e1ca4edbdd service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:53.684225 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:53.684992 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:53.685337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:53.685682 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] PATH_INFO: `/v3/users/93b04d7d46164befaf5595fe0093f516` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:53.686963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b04d7d46164befaf5595fe0093f516)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:53.688905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:53.738455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b04d7d46164befaf5595fe0093f516'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.750934 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b04d7d46164befaf5595fe0093f516'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.770249 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b04d7d46164befaf5595fe0093f516'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.771077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b04d7d46164befaf5595fe0093f516'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.778735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b04d7d46164befaf5595fe0093f516'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:53.788970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1378/2755] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:53 2022] DELETE /identity/v3/users/93b04d7d46164befaf5595fe0093f516 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:54.081494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:54.083244 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:54.100929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cea6df-931c-4798-bd10-ee073bc55408 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:54.145978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:54.161025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:54.161788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:54.162109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:54.162387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] PATH_INFO: `/v3/projects/8b4c36c25d7e4865bde6dc4c239278c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:54.163541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4c36c25d7e4865bde6dc4c239278c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:54.165476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:54.222443 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4c36c25d7e4865bde6dc4c239278c2', 'request_id': 'req-63a12548-0ea9-4e59-9479-7bfd782fd3e1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.228169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4c36c25d7e4865bde6dc4c239278c2', 'request_id': 'req-63a12548-0ea9-4e59-9479-7bfd782fd3e1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.234600 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4c36c25d7e4865bde6dc4c239278c2', 'request_id': 'req-63a12548-0ea9-4e59-9479-7bfd782fd3e1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.248839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] The token cache is being invalidate because project 8b4c36c25d7e4865bde6dc4c239278c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:54.249148 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.249836 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.254229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1378/2756] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:54 2022] DELETE /identity/v3/projects/8b4c36c25d7e4865bde6dc4c239278c2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:54.256957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:54.258659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:54.272863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e68b-ae1d-4bb9-a380-4f558a38ddfe admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:54.280439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:54.349089 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:54.350158 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:54.350487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:54.350763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] PATH_INFO: `/v3/users/2f69e110d4e643b185ba8819677dec74` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:54.352434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f69e110d4e643b185ba8819677dec74)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:54.355598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:54.406928 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f69e110d4e643b185ba8819677dec74'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.415314 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f69e110d4e643b185ba8819677dec74'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.428025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f69e110d4e643b185ba8819677dec74'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.428669 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f69e110d4e643b185ba8819677dec74'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.435496 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-143591cb-8011-4a33-a34f-a997f000eabe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f69e110d4e643b185ba8819677dec74'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.442517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1379/2757] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:07:54 2022] DELETE /identity/v3/users/2f69e110d4e643b185ba8819677dec74 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:54.746851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:54.748586 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:54.761841 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a12548-0ea9-4e59-9479-7bfd782fd3e1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:54.822003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:54.822639 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:54.822923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:54.823206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] PATH_INFO: `/v3/projects/08bedbaf52ad4b9699683ba2d1b03016` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:54.824334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bedbaf52ad4b9699683ba2d1b03016)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:54.826263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:54.875497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bedbaf52ad4b9699683ba2d1b03016', 'request_id': 'req-ef77f0ec-3c78-4655-ac91-8374c494fcc2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.881166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bedbaf52ad4b9699683ba2d1b03016', 'request_id': 'req-ef77f0ec-3c78-4655-ac91-8374c494fcc2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.887707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bedbaf52ad4b9699683ba2d1b03016', 'request_id': 'req-ef77f0ec-3c78-4655-ac91-8374c494fcc2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.900085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] The token cache is being invalidate because project 08bedbaf52ad4b9699683ba2d1b03016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:07:54.900377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.901007 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef77f0ec-3c78-4655-ac91-8374c494fcc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:07:54.902535 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1379/2758] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:07:54 2022] DELETE /identity/v3/projects/08bedbaf52ad4b9699683ba2d1b03016 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:54.907883 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:54.908177 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:54.908456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:54.930795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:54.932077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:54.994238 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1380/2759] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:55.002241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.002532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.002811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.018240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:55.024574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:55.026412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:55.035851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1380/2760] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:55.040828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.051347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007eebf-7eea-4edf-b0de-f265dd40f37a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.056149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.056749 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.057075 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.057283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.057942 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:07:55.058251 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.059834 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.071284 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1381/2761] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:07:55.075622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.085083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d03f8-874f-4a22-9591-9bdcba94f534 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.089631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.090251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.090530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.090801 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.091430 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.093946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.120150 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1381/2762] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:07:55 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:55.124156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.134354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1a30-69d5-42e1-8229-cab3b5e1685d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.144048 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:55.193636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.194270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.194552 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.194829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.195517 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.197522 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.219219 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1382/2763] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:07:55 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:07:55.223574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.233664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d9be-ec59-4624-8f16-b6a7f6287d9b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.238234 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.238825 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.239102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.239479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.240118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.241686 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.252413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1382/2764] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:55.256139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.266161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e28c5c-c9d7-4826-af57-8d87abaf05f9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.270788 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59540a-7726-4dce-9d46-fbd4933802c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.271374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59540a-7726-4dce-9d46-fbd4933802c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.271657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59540a-7726-4dce-9d46-fbd4933802c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.271931 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59540a-7726-4dce-9d46-fbd4933802c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.272567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59540a-7726-4dce-9d46-fbd4933802c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.274179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59540a-7726-4dce-9d46-fbd4933802c3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.285006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1383/2765] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:07:55.288895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.299408 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc5d60-506a-42a2-8d6b-c7aed4ce245b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.304056 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.304694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.304982 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.305259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] PATH_INFO: `/v3/projects/7e0391eedab44e5c8459b5357aadabde/users/39a430af281246a38a29a4aae28a6195/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.319605 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0391eedab44e5c8459b5357aadabde, user_id=39a430af281246a38a29a4aae28a6195, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.322292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.347473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1383/2766] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:07:55 2022] PUT /identity/v3/projects/7e0391eedab44e5c8459b5357aadabde/users/39a430af281246a38a29a4aae28a6195/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:07:55.363164 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.363475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.363758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.400546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] MFA Rules not processed for user `39a430af281246a38a29a4aae28a6195`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:07:55.401734 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:55.479339 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1384/2767] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:07:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:07:55.491647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.493591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:55.552377 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a722-2c2a-480b-b6d9-de9a8b2f0b00 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:55.555909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:55.556520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:55.556797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:55.557065 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:55.564986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:55.566433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:55.574909 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1384/2768] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:55.975119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:55.982145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:56.051623 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:56.068462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a79f6-f19c-4c3a-9af2-08a581ec9f99 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:56.072828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:56.073695 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:56.074097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:56.074385 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:56.076252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:56.156034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:56.157469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:56.175892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1385/2769] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:07:59.327716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:07:59.331581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:59.521555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821d037-2f85-4338-af5b-dc955564276b service glance] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:07:59.527177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:07:59.527854 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:07:59.528140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:07:59.528528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:07:59.531204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:07:59.540263 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:07:59.544379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:07:59.547276 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:07:59.558646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1385/2770] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:07:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:08:02.417312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:02.419462 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:02.433562 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ced3b6-9c98-410f-9993-831271fd2c13 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:02.438452 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:02.439273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:02.439627 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:02.439943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] PATH_INFO: `/v3/users/c8a3f8bef9f0431eac090f8a520f744b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:02.442060 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a3f8bef9f0431eac090f8a520f744b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:02.443893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:02.503701 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a3f8bef9f0431eac090f8a520f744b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.512218 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a3f8bef9f0431eac090f8a520f744b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.528634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a3f8bef9f0431eac090f8a520f744b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.529393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a3f8bef9f0431eac090f8a520f744b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.536975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a3f8bef9f0431eac090f8a520f744b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.543008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1386/2771] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:08:02 2022] DELETE /identity/v3/users/c8a3f8bef9f0431eac090f8a520f744b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:08:02.726507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:02.737707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bab6d7-1e5b-4cf2-8ef6-2da9bacdac9d service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:02.803489 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:02.804158 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:02.804384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:02.804762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] PATH_INFO: `/v3/projects/43e1ec63be6740fa9516a2e5331986a2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:02.806030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e1ec63be6740fa9516a2e5331986a2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:02.810974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:02.858851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e1ec63be6740fa9516a2e5331986a2', 'request_id': 'req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.864806 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e1ec63be6740fa9516a2e5331986a2', 'request_id': 'req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.871544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e1ec63be6740fa9516a2e5331986a2', 'request_id': 'req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.887104 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] The token cache is being invalidate because project 43e1ec63be6740fa9516a2e5331986a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:08:02.887394 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.888017 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:02.890180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1386/2772] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:08:02 2022] DELETE /identity/v3/projects/43e1ec63be6740fa9516a2e5331986a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:08:02.894541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:02.895947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:02.910895 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1deab71-fb5d-4748-a49e-4d8bebf04044 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:02.947340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:08:02.979582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:02.980436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:02.980662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:02.980968 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] PATH_INFO: `/v3/users/f580c5ef6eed4edda4f31758d2cfab69` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:02.982449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f580c5ef6eed4edda4f31758d2cfab69)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:02.984667 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:03.046873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f580c5ef6eed4edda4f31758d2cfab69'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.060865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f580c5ef6eed4edda4f31758d2cfab69'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.074905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f580c5ef6eed4edda4f31758d2cfab69'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.075679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f580c5ef6eed4edda4f31758d2cfab69'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.084145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f580c5ef6eed4edda4f31758d2cfab69'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.094716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1387/2773] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:08:02 2022] DELETE /identity/v3/users/f580c5ef6eed4edda4f31758d2cfab69 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:08:03.275845 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:03.277762 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:03.292541 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e2cc2-ed48-4a31-bc23-32ee754fb2e7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:03.354679 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:03.355312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:03.355629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:03.355876 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] PATH_INFO: `/v3/projects/c9264b7ff15649459f7599559d4fd273` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:03.357015 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9264b7ff15649459f7599559d4fd273)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:03.359454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:03.391123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:08:03.409290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9264b7ff15649459f7599559d4fd273', 'request_id': 'req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.414820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9264b7ff15649459f7599559d4fd273', 'request_id': 'req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.421528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9264b7ff15649459f7599559d4fd273', 'request_id': 'req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.434449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] The token cache is being invalidate because project c9264b7ff15649459f7599559d4fd273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:08:03.434745 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.435395 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b003ff91-a74e-4ae9-aa8d-4a4776bd8dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:03.437544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1387/2774] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:08:03 2022] DELETE /identity/v3/projects/c9264b7ff15649459f7599559d4fd273 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:08:13.087015 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:08:13.431337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:08:26.936675 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:26.938116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:26.950803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02a0f3-0590-4ea7-9521-7f549756fdbd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:27.006415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:27.007045 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:27.007331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:27.007608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] PATH_INFO: `/v3/users/39a430af281246a38a29a4aae28a6195` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:27.008712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a430af281246a38a29a4aae28a6195)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:27.010679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:27.056223 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a430af281246a38a29a4aae28a6195'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.065097 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a430af281246a38a29a4aae28a6195'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.076844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a430af281246a38a29a4aae28a6195'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.077477 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a430af281246a38a29a4aae28a6195'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.084721 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a430af281246a38a29a4aae28a6195'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.090427 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1388/2775] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:08:26 2022] DELETE /identity/v3/users/39a430af281246a38a29a4aae28a6195 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:08:27.095658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:27.096135 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:27.096245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:27.124318 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:08:27.125975 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:27.184497 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1388/2776] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:08:27.194059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:27.195456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:27.272545 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b0802-749e-474e-ba30-9dca1f442d31 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:27.276208 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:27.276830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:27.277113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:27.277386 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:27.286828 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:27.288258 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:27.297557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1389/2777] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:08:27.561595 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:08:27.563193 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:08:27.575470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c782e-2f3d-4759-a52e-9cad5abe7a04 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:08:27.580633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:08:27.581384 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:08:27.581664 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:08:27.581967 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] PATH_INFO: `/v3/projects/7e0391eedab44e5c8459b5357aadabde` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:08:27.583078 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0391eedab44e5c8459b5357aadabde)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:08:27.584977 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:08:27.631518 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0391eedab44e5c8459b5357aadabde', 'request_id': 'req-09b438da-366b-4248-bc3f-76307e7fa47f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.636892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0391eedab44e5c8459b5357aadabde', 'request_id': 'req-09b438da-366b-4248-bc3f-76307e7fa47f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.643499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0391eedab44e5c8459b5357aadabde', 'request_id': 'req-09b438da-366b-4248-bc3f-76307e7fa47f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.656378 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] The token cache is being invalidate because project 7e0391eedab44e5c8459b5357aadabde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:08:27.656670 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.657312 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09b438da-366b-4248-bc3f-76307e7fa47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:08:27.658838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1389/2778] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:08:27 2022] DELETE /identity/v3/projects/7e0391eedab44e5c8459b5357aadabde => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:08:37.293365 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:08:37.653320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:09:08.169548 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:09:08.170984 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:08.239474 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56f99-f023-4251-b097-9cf70d4b7a0d service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:09:08.296175 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:09:08.296774 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:09:08.297055 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:09:08.297330 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:09:08.299092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:08.310543 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:09:08.312115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:09:08.322415 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1390/2779] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:09:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:09:18.318273 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=58 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:09:40.205669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:09:40.206172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:09:40.206275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:09:40.225520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] MFA Rules not processed for user `5a93a2d9266541e7bff502ba074fef5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:09:40.226742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:40.280018 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1390/2780] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:09:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:09:50.273570 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=25 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:09:52.651475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:09:52.652879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:52.707281 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6adc76-6e01-44ab-9682-b65d7d40ea4a service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:09:52.710819 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:09:52.711409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:09:52.711685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:09:52.711955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:09:52.713636 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:52.801341 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:09:52.803340 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2ab2d-2803-4ea4-936e-64edef5730ca service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:09:52.823116 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1391/2781] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:09:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:09:55.209052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:09:55.211092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:55.222772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab796ef2-dc9c-47f5-860a-85bb4bd8a098 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:09:55.226317 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:09:55.226907 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:09:55.227183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:09:55.227450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:09:55.229093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:09:55.241701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:09:55.243137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:09:55.253858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1391/2782] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:09:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:10:02.820213 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=66 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:10:05.249196 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:10:11.194331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:10:11.194867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:10:11.194867 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:10:11.214040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] MFA Rules not processed for user `5d10a2be3765418cb72eeb6b15db5142`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:10:11.215287 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:10:11.268364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1392/2783] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:10:20.204266 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:10:20.213303 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97218cb-36bf-4f66-a7a6-f76b816ba882 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:10:20.217010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:10:20.217701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:10:20.218010 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:10:20.218292 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:10:20.219987 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:10:20.229368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:10:20.230812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f452c2-a669-47ce-a304-616172b1ed6a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:10:20.238629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1392/2784] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:10:21.261912 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=25 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:10:30.235031 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:23.713456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:23.715063 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:23.734413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535b4b0-fb83-42d8-8e8c-4d8926f369cc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:23.744187 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:23.745079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:23.745460 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:23.745833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:23.747869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:23.761505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:23.763432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:23.775959 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1393/2785] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:11:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:11:32.828211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:32.828746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:32.828746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:32.848074 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] MFA Rules not processed for user `5a93a2d9266541e7bff502ba074fef5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:11:32.849280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:32.856091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1393/2786] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:11:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:11:33.693363 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:33.695863 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:33.712857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6cf41-618a-469a-922c-c70da16e4a97 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:33.719731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:33.720809 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:33.721411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:33.721987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:33.723955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:33.740285 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:33.742685 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:33.757884 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1394/2787] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:11:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:11:35.287283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:35.288669 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:35.354993 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd27bd-40c6-406e-aa23-25372de38593 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:35.359557 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:35.360206 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:35.360479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:35.360755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] PATH_INFO: `/v3/users/5d10a2be3765418cb72eeb6b15db5142` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:35.361881 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d10a2be3765418cb72eeb6b15db5142)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:35.364362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:35.418211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d10a2be3765418cb72eeb6b15db5142'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.427163 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d10a2be3765418cb72eeb6b15db5142'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.439480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d10a2be3765418cb72eeb6b15db5142'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.440219 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d10a2be3765418cb72eeb6b15db5142'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.448389 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02206e79-044a-4d77-aefd-51261163fdc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d10a2be3765418cb72eeb6b15db5142'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.454449 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1394/2788] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:11:35 2022] DELETE /identity/v3/users/5d10a2be3765418cb72eeb6b15db5142 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:11:35.638541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:35.653025 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898de92-c3fa-4e0d-aff9-74e0b7913f5c service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:35.727378 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:35.728026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:35.728529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:35.728850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] PATH_INFO: `/v3/projects/a860b6979be5483dba888a37634d196b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:35.730091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a860b6979be5483dba888a37634d196b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:35.733505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:35.804903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a860b6979be5483dba888a37634d196b', 'request_id': 'req-a0014e4a-127b-4d88-b26a-9f5f28ea3316'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.813729 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a860b6979be5483dba888a37634d196b', 'request_id': 'req-a0014e4a-127b-4d88-b26a-9f5f28ea3316'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.821873 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a860b6979be5483dba888a37634d196b', 'request_id': 'req-a0014e4a-127b-4d88-b26a-9f5f28ea3316'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.836059 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] The token cache is being invalidate because project a860b6979be5483dba888a37634d196b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:11:35.836414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.837051 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a0014e4a-127b-4d88-b26a-9f5f28ea3316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:11:35.838553 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1395/2789] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:11:35 2022] DELETE /identity/v3/projects/a860b6979be5483dba888a37634d196b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:11:35.843575 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:35.843868 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:35.844145 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:35.896025 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:11:35.897273 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:35.950763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:35.971976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1395/2790] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:11:35.979673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:35.979967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:35.980247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.000772 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:11:36.002021 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:36.010605 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1396/2791] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:11:36.014916 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.024899 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6ee0b-deb3-4dd9-927a-6caaf6d05e86 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.031402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.032611 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.032900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.033176 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.033830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:11:36.034185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.035824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.056644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1396/2792] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:11:36.061195 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.071024 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee41b5-63ab-45ba-8589-126b3f5376ed None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.075979 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.076573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.076850 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.077126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.077786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.079738 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.122540 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1397/2793] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:11:36 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:11:36.126744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.138824 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9e4bc-d2df-4cde-a180-48bfe5986393 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.218902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.219532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.219813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.220088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.220740 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.223240 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.245123 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1397/2794] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:11:36 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:11:36.249559 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.262608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5077-9439-4bf4-aa25-d44e16bd94a5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.268618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.270163 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.270625 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.271065 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.271878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.273651 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.289855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1398/2795] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:11:36.291213 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.301828 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0acc6c-d341-4ae7-8881-8226bfc76e85 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.308628 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91f656-b169-4bb1-8817-f5a19cf70f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.309256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91f656-b169-4bb1-8817-f5a19cf70f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.309549 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91f656-b169-4bb1-8817-f5a19cf70f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.309827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91f656-b169-4bb1-8817-f5a19cf70f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.310522 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91f656-b169-4bb1-8817-f5a19cf70f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.312117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91f656-b169-4bb1-8817-f5a19cf70f09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.323928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1398/2796] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:11:36.328938 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.339529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44d4e8-9126-4ac2-8f00-c3c0fd7822f4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.344123 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.344766 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.345046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.345318 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] PATH_INFO: `/v3/projects/128503b5d0fe4bb3bb0d5a2f91508024/users/6727ee8ccbb948adbe5a7a962931e58a/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.357481 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:36.360842 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=128503b5d0fe4bb3bb0d5a2f91508024, user_id=6727ee8ccbb948adbe5a7a962931e58a, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.364795 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.385319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1399/2797] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:11:36 2022] PUT /identity/v3/projects/128503b5d0fe4bb3bb0d5a2f91508024/users/6727ee8ccbb948adbe5a7a962931e58a/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:11:36.397274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.397558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.398112 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.428171 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] MFA Rules not processed for user `6727ee8ccbb948adbe5a7a962931e58a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:11:36.429480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:36.495491 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1399/2798] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:11:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:11:36.499953 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.560393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa450566-4d68-4acf-a65e-3c29ef1b84b1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.565161 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.566404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.566684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.566956 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.567621 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.569618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.598140 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1400/2799] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:11:36 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:11:36.602187 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.613634 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c812-79fa-4504-a398-aa1c76b8f70e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.655531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:36.676705 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.677316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.677599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.677874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.678550 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.680454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.702748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1400/2800] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:11:36 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:11:36.706891 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.719471 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5a608-cfe5-4a96-b56a-8a0b0720be4d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.724303 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.724907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.725402 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.726288 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.726943 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.728539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.739777 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1401/2801] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:11:36.744205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.755131 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafa862-78ea-4cec-b5d5-c4487c1f3f46 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.759668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.760309 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.760590 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.760861 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] PATH_INFO: `/v3/projects/2192db51c7dd46af80831b239f94098a/users/514587df9b354643a97134dbe1a5e88b/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.776208 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2192db51c7dd46af80831b239f94098a, user_id=514587df9b354643a97134dbe1a5e88b, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.779016 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.792602 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1401/2802] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:11:36 2022] PUT /identity/v3/projects/2192db51c7dd46af80831b239f94098a/users/514587df9b354643a97134dbe1a5e88b/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:11:36.796580 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.807148 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225403f8-e99b-49d1-8e16-cc671065b6fe admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.866706 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad803d7-4109-40d3-8213-5d4eb6096256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.867647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad803d7-4109-40d3-8213-5d4eb6096256 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.867946 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad803d7-4109-40d3-8213-5d4eb6096256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.868224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad803d7-4109-40d3-8213-5d4eb6096256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.868869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad803d7-4109-40d3-8213-5d4eb6096256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.871449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad803d7-4109-40d3-8213-5d4eb6096256 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.888995 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:36.889506 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1402/2803] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 15:11:36.899940 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03a04a-8ae1-4b43-b449-2bbbf1887e59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:36.905223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:36.906235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.906772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.907156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] PATH_INFO: `/v3/projects/2192db51c7dd46af80831b239f94098a/users/514587df9b354643a97134dbe1a5e88b/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.909624 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2192db51c7dd46af80831b239f94098a, user_id=514587df9b354643a97134dbe1a5e88b, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:36.912643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:36.937704 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1402/2804] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:11:36 2022] PUT /identity/v3/projects/2192db51c7dd46af80831b239f94098a/users/514587df9b354643a97134dbe1a5e88b/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:11:36.948978 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:36.949611 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:36.949933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:36.979483 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] MFA Rules not processed for user `514587df9b354643a97134dbe1a5e88b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:11:36.980434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:37.028630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:37.051420 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1403/2805] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:11:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:11:37.063126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:37.071036 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:37.164885 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd73f14-e7ff-434f-884a-ed976f77f140 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:37.168755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:37.169354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:37.169631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:37.169897 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:37.180581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:37.182454 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:37.192798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1403/2806] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:11:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:11:37.277321 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:37.287903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111b476-1a90-4a3d-a8d2-63a666e1daf7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:37.291579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:37.292276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:37.292613 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:37.292981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:37.359556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:37.361031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:37.370333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1404/2807] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:11:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:11:47.187666 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=59,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:47.365966 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=34 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:11:48.311591 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:48.313469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:48.389574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557bb65-adef-4c23-8660-b81389089a30 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:48.395555 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:48.396152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:48.396437 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:48.396712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:48.398475 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:48.410194 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:48.411750 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:48.422057 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1404/2808] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:11:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:11:52.201478 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:11:52.214581 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3532fc-dfce-46d6-b619-6bf940633294 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:11:52.219905 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc85d332eeaf4c649f51858fa608101f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:11:52.220617 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:11:52.220901 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:11:52.221198 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:11:52.222933 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:11:52.235433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:11:52.237038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46222823-6beb-4a27-b436-ae77c507ab97 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:11:52.247989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1405/2809] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:11:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:11:58.417632 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=34 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:02.243236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:16.936215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:16.997241 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b062-b351-449d-991c-8050ed40234e service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:17.001728 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:17.002373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:17.002658 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:17.002936 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] PATH_INFO: `/v3/users/6727ee8ccbb948adbe5a7a962931e58a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:17.004035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6727ee8ccbb948adbe5a7a962931e58a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:17.005955 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:17.050783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6727ee8ccbb948adbe5a7a962931e58a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.057660 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6727ee8ccbb948adbe5a7a962931e58a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.069172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6727ee8ccbb948adbe5a7a962931e58a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.069804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6727ee8ccbb948adbe5a7a962931e58a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.076542 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6727ee8ccbb948adbe5a7a962931e58a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.082865 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1405/2810] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:16 2022] DELETE /identity/v3/users/6727ee8ccbb948adbe5a7a962931e58a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:17.087502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:17.088018 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:17.088128 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:17.107740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:17.108974 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:17.165638 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1406/2811] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:17.175325 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:17.176721 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:17.233221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d541c7-4080-45d6-a107-a25aeb9727cb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:17.237122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:17.237755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:17.238083 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:17.238397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:17.248215 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:17.249668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:17.259530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1406/2812] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:17.519593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:17.521135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:17.533329 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55280672-c8a0-4949-815f-737f256ecaa9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:17.538074 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:17.538776 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:17.539145 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:17.539492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] PATH_INFO: `/v3/projects/128503b5d0fe4bb3bb0d5a2f91508024` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:17.540688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=128503b5d0fe4bb3bb0d5a2f91508024)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:17.542633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:17.590357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128503b5d0fe4bb3bb0d5a2f91508024', 'request_id': 'req-f485b741-8e92-41d3-9d9a-c5499c7ab92c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.595981 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128503b5d0fe4bb3bb0d5a2f91508024', 'request_id': 'req-f485b741-8e92-41d3-9d9a-c5499c7ab92c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.602224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128503b5d0fe4bb3bb0d5a2f91508024', 'request_id': 'req-f485b741-8e92-41d3-9d9a-c5499c7ab92c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.615576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] The token cache is being invalidate because project 128503b5d0fe4bb3bb0d5a2f91508024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:17.615915 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.616570 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.618619 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1407/2813] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:17 2022] DELETE /identity/v3/projects/128503b5d0fe4bb3bb0d5a2f91508024 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:17.622701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:17.632822 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ad319-4dcf-4841-85bf-4a100e62b8d7 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:17.650331 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:17.689175 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:17.689798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:17.690115 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:17.690396 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] PATH_INFO: `/v3/users/514587df9b354643a97134dbe1a5e88b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:17.692089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] RBAC: Authorizing `identity:delete_user(user_id=514587df9b354643a97134dbe1a5e88b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:17.694038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:17.736880 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514587df9b354643a97134dbe1a5e88b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.746371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514587df9b354643a97134dbe1a5e88b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.760513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514587df9b354643a97134dbe1a5e88b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.761217 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514587df9b354643a97134dbe1a5e88b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.767969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-09f75da5-d3c6-4165-886a-aafc2e31317b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514587df9b354643a97134dbe1a5e88b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:17.773769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1407/2814] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:17 2022] DELETE /identity/v3/users/514587df9b354643a97134dbe1a5e88b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:18.043406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.045027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:18.058373 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485b741-8e92-41d3-9d9a-c5499c7ab92c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.115147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.115800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.116037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.116361 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] PATH_INFO: `/v3/projects/2192db51c7dd46af80831b239f94098a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.117666 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2192db51c7dd46af80831b239f94098a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.119601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.149989 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:18.167455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2192db51c7dd46af80831b239f94098a', 'request_id': 'req-d170f0c7-203b-491a-b812-1998194299ae'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:18.172882 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2192db51c7dd46af80831b239f94098a', 'request_id': 'req-d170f0c7-203b-491a-b812-1998194299ae'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:18.179528 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2192db51c7dd46af80831b239f94098a', 'request_id': 'req-d170f0c7-203b-491a-b812-1998194299ae'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:18.191115 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] The token cache is being invalidate because project 2192db51c7dd46af80831b239f94098a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:18.191412 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:18.192032 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d170f0c7-203b-491a-b812-1998194299ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:18.193475 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1408/2815] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:18 2022] DELETE /identity/v3/projects/2192db51c7dd46af80831b239f94098a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:18.198460 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.198951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.199059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.219203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:18.220547 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:18.284818 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1408/2816] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:18.292718 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.293079 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.293360 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.313241 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:18.314466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:18.322243 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1409/2817] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:18.326280 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.336420 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c0380-b6f8-41db-8fb4-1407fc2b0c85 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.341012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.341599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.341879 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.342180 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.342819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:18.343122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.344732 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.356724 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1409/2818] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:12:18.360839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.370196 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8141d-f113-41e9-b717-758b40c3bb3e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.374754 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.375338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.375618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.375890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.376514 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.378436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.404488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1410/2819] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:18.408480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.419279 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc418cf-0a60-4e2f-8199-5672232ac64b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.444137 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:18.475773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.476366 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.476646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.476919 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.477551 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.479472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.499470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1410/2820] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:12:18.503598 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.513797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6b395-a7eb-4926-b648-542fc78fb5aa admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.518435 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.519529 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.519812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.520087 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.520727 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.522399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.532052 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1411/2821] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:18.536352 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.546478 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea399e-9803-4edc-bb3d-381600973777 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.551258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc505e-2a3b-4f2e-9dcd-134dd8714b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.551851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc505e-2a3b-4f2e-9dcd-134dd8714b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.552140 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc505e-2a3b-4f2e-9dcd-134dd8714b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.552414 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc505e-2a3b-4f2e-9dcd-134dd8714b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.553055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc505e-2a3b-4f2e-9dcd-134dd8714b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.554681 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc505e-2a3b-4f2e-9dcd-134dd8714b7f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.568707 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1411/2822] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:18.573125 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.583957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dbd92-0f73-48da-9dca-14fd784f5240 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.588786 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.589456 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.589737 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.590147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] PATH_INFO: `/v3/projects/8f85bb6b2ed24161b06dc2a2520d8858/users/fd8314ff5fd748608f1fd977ae9fcddd/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.608593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f85bb6b2ed24161b06dc2a2520d8858, user_id=fd8314ff5fd748608f1fd977ae9fcddd, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.611277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.623967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1412/2823] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:18 2022] PUT /identity/v3/projects/8f85bb6b2ed24161b06dc2a2520d8858/users/fd8314ff5fd748608f1fd977ae9fcddd/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:18.634245 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.634533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.634830 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.663068 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] MFA Rules not processed for user `fd8314ff5fd748608f1fd977ae9fcddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:18.664252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:18.725290 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1412/2824] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:12:18.729634 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.789673 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c8071-a331-4f86-98a9-fe5b972c9a21 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.794408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.795001 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.795280 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.795546 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.796183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.798093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.823260 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1413/2825] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:18.827494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.838748 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912aec65-41c5-4ab5-afc7-9f5d88401d75 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.896379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.896900 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.897184 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.897496 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.898155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.900072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.920629 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1413/2826] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:18 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:18.924688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.935157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7936aa-dc70-472e-8f31-989971189b05 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.939869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.940975 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.941319 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.941601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.942271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.943879 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:18.955839 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1414/2827] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:18.960749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:18.970659 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7800736-651f-429c-9ea8-05bedf01dda7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:18.975159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:18.975790 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:18.976070 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:18.976343 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] PATH_INFO: `/v3/projects/c29f53560b1746bb8e1cd27c5309a729/users/3352ea35943a4600941b337a4e64f4c3/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:18.990423 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29f53560b1746bb8e1cd27c5309a729, user_id=3352ea35943a4600941b337a4e64f4c3, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:18.993320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:19.006353 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1414/2828] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:18 2022] PUT /identity/v3/projects/c29f53560b1746bb8e1cd27c5309a729/users/3352ea35943a4600941b337a4e64f4c3/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:19.010154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:19.020683 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca071-58b7-4f5c-b1ce-97c21b0dd9d7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:19.040530 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:19.078006 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9c3e3-e82f-4936-a480-c204e813e3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:19.078594 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9c3e3-e82f-4936-a480-c204e813e3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:19.078869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9c3e3-e82f-4936-a480-c204e813e3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:19.079136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9c3e3-e82f-4936-a480-c204e813e3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:19.079756 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9c3e3-e82f-4936-a480-c204e813e3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:19.081350 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9c3e3-e82f-4936-a480-c204e813e3f4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:19.091046 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1415/2829] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:19.095127 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:19.100578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:19.106775 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90038-328d-476f-84e8-37f4dc9da888 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:19.111564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:19.112205 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:19.112978 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:19.113259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] PATH_INFO: `/v3/projects/c29f53560b1746bb8e1cd27c5309a729/users/3352ea35943a4600941b337a4e64f4c3/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:19.115204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29f53560b1746bb8e1cd27c5309a729, user_id=3352ea35943a4600941b337a4e64f4c3, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:19.118121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:19.128737 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1415/2830] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:19 2022] PUT /identity/v3/projects/c29f53560b1746bb8e1cd27c5309a729/users/3352ea35943a4600941b337a4e64f4c3/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:19.139770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:19.140276 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:19.140384 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:19.167707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] MFA Rules not processed for user `3352ea35943a4600941b337a4e64f4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:19.168844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:19.234665 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1416/2831] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:19.246236 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:19.247942 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:19.304647 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51360ed-ef0d-4e75-a871-ec4dbd5870af admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:19.308259 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:19.308856 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:19.309141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:19.309412 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:19.318369 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:19.319782 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:19.328961 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1416/2832] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:19.411887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:19.423855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0e287-6275-4089-805d-9c75e4792eed None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:19.427501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aad213-b9e5-4264-b43e-52c7a898ec76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:19.428092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aad213-b9e5-4264-b43e-52c7a898ec76 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:19.428370 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aad213-b9e5-4264-b43e-52c7a898ec76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:19.428645 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aad213-b9e5-4264-b43e-52c7a898ec76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:19.484643 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aad213-b9e5-4264-b43e-52c7a898ec76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:19.486152 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aad213-b9e5-4264-b43e-52c7a898ec76 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:19.495149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1417/2833] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:22.202267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:22.263872 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929f719-5e9b-40ff-88c7-3015b3760b32 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:22.269235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:22.269851 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:22.270169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:22.270447 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] PATH_INFO: `/v3/users/fd8314ff5fd748608f1fd977ae9fcddd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:22.271563 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8314ff5fd748608f1fd977ae9fcddd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:22.273466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:22.320256 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8314ff5fd748608f1fd977ae9fcddd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.327075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8314ff5fd748608f1fd977ae9fcddd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.339774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8314ff5fd748608f1fd977ae9fcddd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.340424 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8314ff5fd748608f1fd977ae9fcddd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.348059 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8314ff5fd748608f1fd977ae9fcddd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.354458 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1417/2834] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:22 2022] DELETE /identity/v3/users/fd8314ff5fd748608f1fd977ae9fcddd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:22.359064 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:22.359538 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:22.359660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:22.378310 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:22.379500 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:22.418866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:22.442136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1418/2835] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:22.451914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:22.453399 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:22.468652 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:22.520902 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797cdd-3cbd-4e9a-a289-d4641f6cf1d6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:22.524567 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:22.525169 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:22.525450 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:22.525722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:22.535536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:22.536976 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:22.546813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1418/2836] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 15:12:22.809812 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:22.811377 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:22.825237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016d55c-5049-4b34-beb9-50d153408992 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:22.829896 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:22.830689 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:22.831054 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:22.831357 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] PATH_INFO: `/v3/projects/8f85bb6b2ed24161b06dc2a2520d8858` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:22.832525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f85bb6b2ed24161b06dc2a2520d8858)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:22.834392 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:22.881312 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f85bb6b2ed24161b06dc2a2520d8858', 'request_id': 'req-d3389098-c024-4057-bfbc-5645ab42aa33'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.886800 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f85bb6b2ed24161b06dc2a2520d8858', 'request_id': 'req-d3389098-c024-4057-bfbc-5645ab42aa33'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.892700 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f85bb6b2ed24161b06dc2a2520d8858', 'request_id': 'req-d3389098-c024-4057-bfbc-5645ab42aa33'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.903789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] The token cache is being invalidate because project 8f85bb6b2ed24161b06dc2a2520d8858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:22.904077 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.904686 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:22.906150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1419/2837] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:22 2022] DELETE /identity/v3/projects/8f85bb6b2ed24161b06dc2a2520d8858 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:22.910172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:22.919643 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b348f1-0d4d-475e-bec7-89ee3268c99d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:22.976327 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:22.976958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:22.977243 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:22.977523 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] PATH_INFO: `/v3/users/3352ea35943a4600941b337a4e64f4c3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:22.978675 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3352ea35943a4600941b337a4e64f4c3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:22.980594 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.024585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3352ea35943a4600941b337a4e64f4c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.031226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3352ea35943a4600941b337a4e64f4c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.043546 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3352ea35943a4600941b337a4e64f4c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.044156 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3352ea35943a4600941b337a4e64f4c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.051405 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-376d7cfe-f095-4d27-92b1-d9e76d41431c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3352ea35943a4600941b337a4e64f4c3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.057050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1419/2838] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:22 2022] DELETE /identity/v3/users/3352ea35943a4600941b337a4e64f4c3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:23.335103 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.337081 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:23.350487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389098-c024-4057-bfbc-5645ab42aa33 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.410165 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.410822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.411061 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.411332 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] PATH_INFO: `/v3/projects/c29f53560b1746bb8e1cd27c5309a729` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.412449 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29f53560b1746bb8e1cd27c5309a729)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.414408 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.462451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29f53560b1746bb8e1cd27c5309a729', 'request_id': 'req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.467830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29f53560b1746bb8e1cd27c5309a729', 'request_id': 'req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.473725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29f53560b1746bb8e1cd27c5309a729', 'request_id': 'req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.481890 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:23.487159 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] The token cache is being invalidate because project c29f53560b1746bb8e1cd27c5309a729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:23.487616 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.488746 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-693fa5f1-f4f6-427a-bfb5-4b747fa5372c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:23.490366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1420/2839] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:23 2022] DELETE /identity/v3/projects/c29f53560b1746bb8e1cd27c5309a729 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:23.495574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.496045 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.496160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.515951 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:23.517081 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:23.545688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:23.577099 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1420/2840] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:23.585422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.585728 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.586038 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.605977 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:23.607362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:23.615739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1421/2841] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:23.620270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.629840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d073e-f539-4ccf-a4de-ad008e4aabd3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.634415 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.635005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.635287 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.635562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.636204 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:23.636533 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.638149 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.650073 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1421/2842] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:12:23.654414 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.665620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810ecf-3337-4ec7-bf72-cb56741a3374 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.670209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.670797 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.671073 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.671345 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.671961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.673876 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.699907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1422/2843] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:23 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:23.704299 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.716562 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4f08e-888a-464e-892a-ffffc1c1f905 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.775165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.775683 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.775959 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.776230 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.776848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.778773 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.799218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1422/2844] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:23 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:23.803406 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.813501 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64fe05-7939-4f2b-8d8b-8383c03ee043 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.818138 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.818725 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.819003 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.819271 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.819903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.821490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.831696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1423/2845] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:23.835513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.845075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1af86-81a1-40a1-ae5e-4638ea0758f4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.850255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2ade-c842-440d-90e6-cb32095c3fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.850829 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2ade-c842-440d-90e6-cb32095c3fab admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.851108 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2ade-c842-440d-90e6-cb32095c3fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.851379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2ade-c842-440d-90e6-cb32095c3fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.852005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad2ade-c842-440d-90e6-cb32095c3fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.853566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad2ade-c842-440d-90e6-cb32095c3fab admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.863933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1423/2846] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:23.868197 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:23.878134 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b941-cca1-4847-8a60-809044f38c52 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:23.882871 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:23.883512 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.883794 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.884072 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] PATH_INFO: `/v3/projects/88581d3cb9724f13a17bd19f3b23b3b5/users/bb1f1c634b2a490280e27ad3ee771050/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:23.898660 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88581d3cb9724f13a17bd19f3b23b3b5, user_id=bb1f1c634b2a490280e27ad3ee771050, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:23.901305 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:23.914282 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1424/2847] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:23 2022] PUT /identity/v3/projects/88581d3cb9724f13a17bd19f3b23b3b5/users/bb1f1c634b2a490280e27ad3ee771050/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:23.978587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:23.978893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:23.979167 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.007085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] MFA Rules not processed for user `bb1f1c634b2a490280e27ad3ee771050`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:24.008275 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:24.069000 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1424/2848] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:24.073556 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.134343 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd120d-43f5-489e-92ef-28d8fe50eb87 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.138983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.139582 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.139864 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.140142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.140763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.142698 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.168596 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1425/2849] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:24 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:24.172697 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.183109 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46beddb3-7e44-4677-827a-b1162c4e5dbd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.208774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:24.239419 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.240023 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.240306 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.240581 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.241216 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.243416 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.263974 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1425/2850] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:24 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:12:24.267922 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.278602 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea1d9d-7d82-40d7-ac09-fd98d82c2c01 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.283220 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.283808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.284092 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.284388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.285027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.286662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.296902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1426/2851] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:24.301397 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.311082 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e4d8-addb-4e63-9c1b-0abdd9738d03 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.316122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.316747 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.317026 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.317298 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] PATH_INFO: `/v3/projects/02de69b46fec460590af28e46769e684/users/c0095dd8e30c402f8f9c59d0c69d7a8a/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.331648 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02de69b46fec460590af28e46769e684, user_id=c0095dd8e30c402f8f9c59d0c69d7a8a, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.334258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.346813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1426/2852] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:24 2022] PUT /identity/v3/projects/02de69b46fec460590af28e46769e684/users/c0095dd8e30c402f8f9c59d0c69d7a8a/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:12:24.350889 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.360833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f65bd-1700-4310-81ad-9083c24f853f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.415347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f32e6-7ce2-4317-b204-d87a0c5ebe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.416016 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f32e6-7ce2-4317-b204-d87a0c5ebe77 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.416214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f32e6-7ce2-4317-b204-d87a0c5ebe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.416488 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f32e6-7ce2-4317-b204-d87a0c5ebe77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.417117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f32e6-7ce2-4317-b204-d87a0c5ebe77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.418742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f32e6-7ce2-4317-b204-d87a0c5ebe77 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.429008 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1427/2853] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:24.433320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.443904 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173bfaa-8e03-4e77-a9c2-1d0a8582eb91 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.448653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.449297 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.449578 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.449855 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] PATH_INFO: `/v3/projects/02de69b46fec460590af28e46769e684/users/c0095dd8e30c402f8f9c59d0c69d7a8a/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.451848 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02de69b46fec460590af28e46769e684, user_id=c0095dd8e30c402f8f9c59d0c69d7a8a, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.454613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.464593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1427/2854] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:24 2022] PUT /identity/v3/projects/02de69b46fec460590af28e46769e684/users/c0095dd8e30c402f8f9c59d0c69d7a8a/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:24.474204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a518b-d469-431c-a5a0-560e9123532a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.474490 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a518b-d469-431c-a5a0-560e9123532a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.474762 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a518b-d469-431c-a5a0-560e9123532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.482629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:24.504443 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-280a518b-d469-431c-a5a0-560e9123532a None None] MFA Rules not processed for user `c0095dd8e30c402f8f9c59d0c69d7a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:24.505646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-280a518b-d469-431c-a5a0-560e9123532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:24.570355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1428/2855] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:24.582736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.584336 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:24.641895 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6815def-1233-41f5-9e2e-6cb65c1ca402 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.645532 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.646152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.646433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.646702 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.655480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.656891 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:24.666531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1428/2856] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:24.973413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a518b-d469-431c-a5a0-560e9123532a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:24.982356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a518b-d469-431c-a5a0-560e9123532a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:24.986899 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0095dd8e30c402f8f9c59d0c69d7a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02de69b46fec460590af28e46769e684', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:24.988042 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:24.988325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:24.988599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:24.989236 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:24.989542 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:24.991178 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:25.002469 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1429/2857] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:12:25.006368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:25.015371 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea99b8-8e21-4dc8-a799-3577f470dfa6 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:25.019920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0095dd8e30c402f8f9c59d0c69d7a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02de69b46fec460590af28e46769e684', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:25.020506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:25.020794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:25.021072 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:25.021695 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:25.023622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:25.049442 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1429/2858] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:25 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:25.054162 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:25.063892 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24835cd2-d8d1-4e9c-82ea-835177e56e1a tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:25.118955 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0095dd8e30c402f8f9c59d0c69d7a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02de69b46fec460590af28e46769e684', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:25.119555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:25.119837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:25.120113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:25.120748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:25.121053 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:25.122677 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:25.135068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1430/2859] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:12:25.171796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:25.181712 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a929ba1-9f2c-40e1-801d-70eddca43ec9 tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:25.186188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0095dd8e30c402f8f9c59d0c69d7a8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02de69b46fec460590af28e46769e684', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:25.186796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:25.187075 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:25.187346 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] PATH_INFO: `/v3/projects/2aa427fd78db40baa67390ccc7485335` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:25.188479 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2aa427fd78db40baa67390ccc7485335)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:25.190402 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:25.234673 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa427fd78db40baa67390ccc7485335', 'request_id': 'req-45e13fed-6152-4f39-854e-b3f5400672bd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.239736 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:25.241165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa427fd78db40baa67390ccc7485335', 'request_id': 'req-45e13fed-6152-4f39-854e-b3f5400672bd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.249510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa427fd78db40baa67390ccc7485335', 'request_id': 'req-45e13fed-6152-4f39-854e-b3f5400672bd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.262107 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] The token cache is being invalidate because project 2aa427fd78db40baa67390ccc7485335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:25.262427 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.263091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-45e13fed-6152-4f39-854e-b3f5400672bd tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.264579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1430/2860] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:25 2022] DELETE /identity/v3/projects/2aa427fd78db40baa67390ccc7485335 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:25.454068 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:25.456086 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:25.518534 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c922ff-9a56-4ee2-893e-238da7e3cb1d tempest-VolumeQuotaClassesTest-2045027047 tempest-VolumeQuotaClassesTest-2045027047-project-admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:25.530237 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:25.572770 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:25.573393 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:25.573678 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:25.573983 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] PATH_INFO: `/v3/users/bb1f1c634b2a490280e27ad3ee771050` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:25.575113 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1f1c634b2a490280e27ad3ee771050)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:25.577033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:25.621897 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1f1c634b2a490280e27ad3ee771050'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.630275 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1f1c634b2a490280e27ad3ee771050'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.641988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1f1c634b2a490280e27ad3ee771050'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.642629 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1f1c634b2a490280e27ad3ee771050'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.649857 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1f1c634b2a490280e27ad3ee771050'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:25.656404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1431/2861] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:25 2022] DELETE /identity/v3/users/bb1f1c634b2a490280e27ad3ee771050 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:25.660886 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:25.661354 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:25.661480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:25.679587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:25.680771 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:25.736920 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1431/2862] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:25.746787 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:25.748230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:25.804807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c21723-d483-4eb8-80d3-fa6395b73c2d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:25.808404 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:25.808993 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:25.809267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:25.809533 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:25.818290 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:25.819713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:25.828722 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1432/2863] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:26.099490 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:26.100958 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:26.113267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ce19e-6c99-4c02-aa52-0c177574955e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:26.119269 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:26.120116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.120513 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.120797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] PATH_INFO: `/v3/projects/88581d3cb9724f13a17bd19f3b23b3b5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.122111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88581d3cb9724f13a17bd19f3b23b3b5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:26.128615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:26.176126 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88581d3cb9724f13a17bd19f3b23b3b5', 'request_id': 'req-c6474f81-664d-406e-80ea-08bc3e160c0f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.181802 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88581d3cb9724f13a17bd19f3b23b3b5', 'request_id': 'req-c6474f81-664d-406e-80ea-08bc3e160c0f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.187956 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88581d3cb9724f13a17bd19f3b23b3b5', 'request_id': 'req-c6474f81-664d-406e-80ea-08bc3e160c0f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.199986 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] The token cache is being invalidate because project 88581d3cb9724f13a17bd19f3b23b3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:26.200277 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.200898 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.202409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1432/2864] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:26 2022] DELETE /identity/v3/projects/88581d3cb9724f13a17bd19f3b23b3b5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:26.206459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:26.215995 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b09d-cd67-4642-a54a-01e6a7aca07c service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:26.253557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:26.272967 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:26.273527 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.273806 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.274108 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] PATH_INFO: `/v3/users/c0095dd8e30c402f8f9c59d0c69d7a8a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.275214 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0095dd8e30c402f8f9c59d0c69d7a8a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:26.277135 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:26.320200 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0095dd8e30c402f8f9c59d0c69d7a8a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.327317 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0095dd8e30c402f8f9c59d0c69d7a8a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.339525 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0095dd8e30c402f8f9c59d0c69d7a8a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.340156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0095dd8e30c402f8f9c59d0c69d7a8a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.346843 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4e33282a-8198-4c39-a409-f9f69a453787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0095dd8e30c402f8f9c59d0c69d7a8a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.352764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1433/2865] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:26 2022] DELETE /identity/v3/users/c0095dd8e30c402f8f9c59d0c69d7a8a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:26.616966 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:26.618433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:26.631592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6474f81-664d-406e-80ea-08bc3e160c0f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:26.687095 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:26.687774 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.687984 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.688251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] PATH_INFO: `/v3/projects/02de69b46fec460590af28e46769e684` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.689337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02de69b46fec460590af28e46769e684)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:26.691268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:26.701132 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:26.737289 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02de69b46fec460590af28e46769e684', 'request_id': 'req-a432d2da-2854-4e8c-a992-68df485f0835'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.743035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02de69b46fec460590af28e46769e684', 'request_id': 'req-a432d2da-2854-4e8c-a992-68df485f0835'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.749046 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02de69b46fec460590af28e46769e684', 'request_id': 'req-a432d2da-2854-4e8c-a992-68df485f0835'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.760963 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] The token cache is being invalidate because project 02de69b46fec460590af28e46769e684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:26.761255 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.761873 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a432d2da-2854-4e8c-a992-68df485f0835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:26.763363 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1433/2866] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:26 2022] DELETE /identity/v3/projects/02de69b46fec460590af28e46769e684 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:26.769640 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.770126 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.770224 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.790100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:26.791309 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:26.848932 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1434/2867] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:26.856530 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.856820 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.857097 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.876744 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:26.878003 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:26.885765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1434/2868] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:26.889964 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:26.900122 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fd39e-a5e7-4140-a8bf-298cb7467f66 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:26.904724 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:26.905316 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.905597 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.905869 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.906536 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:26.906838 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:26.908434 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:26.920902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1435/2869] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 15:12:26.925160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:26.934334 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef40e-45a4-49f8-9446-4d84581789fa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:26.938968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:26.939559 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:26.939839 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:26.940116 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:26.941262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:26.943183 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:26.970122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1435/2870] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:26 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:26.974229 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:26.984119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac0d75-7d4a-474a-babd-47af4ec6a168 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:27.027170 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:27.041102 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:27.041714 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:27.042023 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:27.042296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:27.042923 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:27.044841 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:27.065712 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1436/2871] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:26 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:27.069795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:27.079893 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f6da5-53a0-4bbf-8cc5-86df5bdf9d1e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:27.085128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:27.085746 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:27.086062 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:27.086337 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:27.086968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:27.088544 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:27.098584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1436/2872] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:27.102950 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:27.113648 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835af8b-23b2-4773-b257-03f46824df80 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:27.119826 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f324f-ad0f-4a02-8125-812f563261f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:27.120429 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f324f-ad0f-4a02-8125-812f563261f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:27.120713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f324f-ad0f-4a02-8125-812f563261f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:27.120988 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f324f-ad0f-4a02-8125-812f563261f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:27.121633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f324f-ad0f-4a02-8125-812f563261f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:27.123266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f324f-ad0f-4a02-8125-812f563261f8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:27.134655 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1437/2873] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:27.138796 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:27.149495 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541444f4-0216-4e37-ac02-43d82615ea24 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:27.154338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:27.155043 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:27.155332 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:27.155613 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] PATH_INFO: `/v3/projects/8e17134537a343bf94a964a4d461bddc/users/87665bdcd7f0457cab2a012621cd3aa4/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:27.170425 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e17134537a343bf94a964a4d461bddc, user_id=87665bdcd7f0457cab2a012621cd3aa4, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:27.173071 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:27.185742 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1437/2874] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:27 2022] PUT /identity/v3/projects/8e17134537a343bf94a964a4d461bddc/users/87665bdcd7f0457cab2a012621cd3aa4/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:27.197618 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:27.197929 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:27.198209 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:27.225659 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] MFA Rules not processed for user `87665bdcd7f0457cab2a012621cd3aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:27.226916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:27.291986 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1438/2875] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:27.303641 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:27.305288 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:27.361548 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de4222-7865-4747-b636-3b6f4effcac1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:27.365102 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc636e02-7af6-4357-ba03-809947202efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:27.365684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636e02-7af6-4357-ba03-809947202efb service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:27.365992 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636e02-7af6-4357-ba03-809947202efb service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:27.366268 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc636e02-7af6-4357-ba03-809947202efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:27.374577 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc636e02-7af6-4357-ba03-809947202efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:27.375991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc636e02-7af6-4357-ba03-809947202efb service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:27.384598 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1438/2876] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:37.289129 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:37.380760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:45.979822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.038758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b2b4b-0e56-48b8-ae20-b0d1be2063d3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.043342 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.043961 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.044235 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.044505 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] PATH_INFO: `/v3/users/87665bdcd7f0457cab2a012621cd3aa4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.045601 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=87665bdcd7f0457cab2a012621cd3aa4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.047550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:46.092866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87665bdcd7f0457cab2a012621cd3aa4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.099980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87665bdcd7f0457cab2a012621cd3aa4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.112191 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87665bdcd7f0457cab2a012621cd3aa4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.112784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87665bdcd7f0457cab2a012621cd3aa4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.120338 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87665bdcd7f0457cab2a012621cd3aa4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.126297 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1439/2877] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:45 2022] DELETE /identity/v3/users/87665bdcd7f0457cab2a012621cd3aa4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:46.131487 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.131994 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.132114 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.155251 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:46.156459 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:46.213387 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1439/2878] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 15:12:46.223331 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.224708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:46.280775 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb611e-a726-4398-83e2-97bcf4555f5b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.284390 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f469-0815-416f-9212-52ccd72db8e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.284987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f469-0815-416f-9212-52ccd72db8e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.285267 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f469-0815-416f-9212-52ccd72db8e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.285630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f469-0815-416f-9212-52ccd72db8e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.294356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1f469-0815-416f-9212-52ccd72db8e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.295773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1f469-0815-416f-9212-52ccd72db8e6 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:46.305419 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1440/2879] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:46.567498 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.568968 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:46.583682 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd290a-bcb6-4d7c-b69e-5086255ade80 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.588380 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.589039 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.589320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.589592 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] PATH_INFO: `/v3/projects/8e17134537a343bf94a964a4d461bddc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.590784 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e17134537a343bf94a964a4d461bddc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.592752 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:46.639122 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e17134537a343bf94a964a4d461bddc', 'request_id': 'req-462fc708-0226-47e1-ae90-69727e0710a8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.644531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e17134537a343bf94a964a4d461bddc', 'request_id': 'req-462fc708-0226-47e1-ae90-69727e0710a8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.650710 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e17134537a343bf94a964a4d461bddc', 'request_id': 'req-462fc708-0226-47e1-ae90-69727e0710a8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.663316 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] The token cache is being invalidate because project 8e17134537a343bf94a964a4d461bddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:46.663607 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.664227 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-462fc708-0226-47e1-ae90-69727e0710a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:46.665688 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1440/2880] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:46 2022] DELETE /identity/v3/projects/8e17134537a343bf94a964a4d461bddc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:12:46.671066 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.671541 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.671803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.691829 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:46.693035 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:46.709270 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:46.754740 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1441/2881] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:46.762474 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.762763 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.763041 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.783159 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:46.784585 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:46.794831 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1441/2882] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:46.799230 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.808852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c070d-8a24-4515-b09f-07b91267e93b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.813799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.814422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.814696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.814963 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.815599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:46.815898 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.817524 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:46.828789 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1442/2883] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:12:46.833516 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.843050 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae21e-916e-493c-b0fb-d43f0a5a2b88 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.848411 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.849038 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.849321 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.849593 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.850265 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.852155 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:46.878749 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1442/2884] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:46 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:12:46.882866 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.892579 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd45e4c-bfed-406f-ad0e-aed2747138b2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.947732 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.948333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.948607 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.948874 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.949486 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.951413 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:46.972154 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1443/2885] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:46 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:46.977092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:46.987221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65ac58-8474-4d6a-b2e4-506872e2b94b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:46.992379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:46.992973 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:46.993258 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:46.993558 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:46.994229 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:46.995812 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.005588 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1443/2886] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:47.009394 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.019004 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ef8ee-ada3-4188-a3a3-5adcc94f780e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.024033 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdef0a0-0e62-4265-b5c6-30bd37adb4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.024608 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdef0a0-0e62-4265-b5c6-30bd37adb4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.024902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdef0a0-0e62-4265-b5c6-30bd37adb4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.025155 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdef0a0-0e62-4265-b5c6-30bd37adb4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.025782 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdef0a0-0e62-4265-b5c6-30bd37adb4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.027381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdef0a0-0e62-4265-b5c6-30bd37adb4c7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.038348 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1444/2887] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:47.042118 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.051769 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5c66f-510d-460a-9635-bae482bdc585 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.056315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.056941 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.057226 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.057499 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] PATH_INFO: `/v3/projects/75de5634834e4dbeaed305d39608d86e/users/8fc7add8f1b24eb8ba1fd6176573a61d/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.073209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75de5634834e4dbeaed305d39608d86e, user_id=8fc7add8f1b24eb8ba1fd6176573a61d, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.075935 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.089172 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1444/2888] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:47 2022] PUT /identity/v3/projects/75de5634834e4dbeaed305d39608d86e/users/8fc7add8f1b24eb8ba1fd6176573a61d/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 15:12:47.101567 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.101858 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.102298 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.141259 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] MFA Rules not processed for user `8fc7add8f1b24eb8ba1fd6176573a61d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:47.142502 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:47.209228 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1445/2889] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:47.213622 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.234128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:47.277684 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03e18b-484d-47cd-acee-67fba80db515 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.282633 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.283223 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.283507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.283783 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.284526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.286569 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.312274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1445/2890] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:47 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:47.316827 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.327523 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bc565-f447-494f-8e29-5da5d5caa940 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.340036 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:47.384742 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.385422 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.385709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.386012 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.386633 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.388550 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.409359 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1446/2891] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:47 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:47.413410 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.423393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f1791-058b-4623-a164-e3a46ddde329 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.428703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.429284 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.429566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.429840 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.430520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.432076 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.443161 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1446/2892] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:47.447327 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.461263 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52625e-ee3c-4856-be06-07e685bb0db6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.465825 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c9f8d-a1c1-4ba2-8d0f-5c9f2189309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.466451 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c9f8d-a1c1-4ba2-8d0f-5c9f2189309e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.466735 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c9f8d-a1c1-4ba2-8d0f-5c9f2189309e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.467009 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c9f8d-a1c1-4ba2-8d0f-5c9f2189309e admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.467646 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c9f8d-a1c1-4ba2-8d0f-5c9f2189309e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.469247 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c9f8d-a1c1-4ba2-8d0f-5c9f2189309e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.479157 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1447/2893] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:47.483190 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.493864 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a2918-a5d5-4566-897e-34dc4468fc08 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.498574 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.499250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.499536 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.499815 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] PATH_INFO: `/v3/projects/666dc42a35cb4008aa91d31209a5da20/users/96553802c82047979e12215dda20c2f9/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.516315 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] RBAC: Authorizing `identity:create_grant(project_id=666dc42a35cb4008aa91d31209a5da20, user_id=96553802c82047979e12215dda20c2f9, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.519011 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.531615 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1447/2894] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:47 2022] PUT /identity/v3/projects/666dc42a35cb4008aa91d31209a5da20/users/96553802c82047979e12215dda20c2f9/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:47.541947 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.542336 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.542620 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.570628 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] MFA Rules not processed for user `96553802c82047979e12215dda20c2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:47.571865 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:47.635029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1448/2895] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:47.639656 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.700020 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a95e8-6165-4f51-92b0-8c1db763595e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.704644 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.705235 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.705521 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.705798 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.706476 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.708438 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.734203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1448/2896] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:47 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:47.738731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.750833 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a66419-6d22-4cbe-bc31-4dfb489fe252 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.809217 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.809830 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.810137 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.810409 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.811029 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.812940 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.834355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1449/2897] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:47 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:47.838473 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.848564 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427cdeb-346b-428a-b337-e3af5fe9ac35 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.853052 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.853631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.853928 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.854211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.854844 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.856417 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.867421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1449/2898] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:47.871391 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.881814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3c150-e1ad-44b4-91a2-c577f25f2714 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.886448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.887088 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.887371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.887647 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] PATH_INFO: `/v3/projects/5216685e5d304d52a24c1e08e419ac1c/users/d9962d5891a14d5f830c830a8504dffc/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.899447 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:47.902600 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5216685e5d304d52a24c1e08e419ac1c, user_id=d9962d5891a14d5f830c830a8504dffc, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.905719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:47.919107 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1450/2899] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:47 2022] PUT /identity/v3/projects/5216685e5d304d52a24c1e08e419ac1c/users/d9962d5891a14d5f830c830a8504dffc/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:47.923701 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:47.934510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b322d3-e486-40b7-bb39-37133ff6bbd3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:47.975085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:47.992113 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1116a-90a8-44b0-9c74-e002577e84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:47.992677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1116a-90a8-44b0-9c74-e002577e84b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:47.992971 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1116a-90a8-44b0-9c74-e002577e84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:47.993270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1116a-90a8-44b0-9c74-e002577e84b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:47.993933 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1116a-90a8-44b0-9c74-e002577e84b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:47.995654 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1116a-90a8-44b0-9c74-e002577e84b9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:48.005328 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1450/2900] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:48.009492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:48.019465 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f7bd4-3758-4c8d-8b6f-90774fda3e15 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:48.024326 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:48.024688 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:48.025026 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:48.025296 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] PATH_INFO: `/v3/projects/5216685e5d304d52a24c1e08e419ac1c/users/d9962d5891a14d5f830c830a8504dffc/roles/7f15657ea9dc4fa986324ff0aa9671ed` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:48.028149 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5216685e5d304d52a24c1e08e419ac1c, user_id=d9962d5891a14d5f830c830a8504dffc, role_id=7f15657ea9dc4fa986324ff0aa9671ed)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:48.030255 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:48.041333 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1451/2901] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:48 2022] PUT /identity/v3/projects/5216685e5d304d52a24c1e08e419ac1c/users/d9962d5891a14d5f830c830a8504dffc/roles/7f15657ea9dc4fa986324ff0aa9671ed => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:48.052320 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:48.052617 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:48.052924 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:48.081022 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] MFA Rules not processed for user `d9962d5891a14d5f830c830a8504dffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:48.082262 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:48.148359 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1451/2902] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:48.159448 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:48.161604 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:48.216254 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4464a-7d3e-443c-be0c-14d4fac58e6a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:48.219852 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:48.220436 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:48.220713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:48.220982 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:48.275204 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:48.276557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:48.285763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1452/2903] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:49.785816 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:49.787440 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:49.799760 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83055e-64e9-4532-aa2d-6f05e54667de None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:49.803375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:49.803969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:49.804250 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:49.804528 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:49.858456 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:49.859811 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:49.869264 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1452/2904] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:50.393957 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:50.403356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1d651-3f39-4fb7-8198-91f2cd0b1d7c service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:50.406902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85671b0-c3c7-4abe-b814-9464cd9e8ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83758a542c2e4314871bf03fd2cca8b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:50.407495 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85671b0-c3c7-4abe-b814-9464cd9e8ccb service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:50.407773 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85671b0-c3c7-4abe-b814-9464cd9e8ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:50.408043 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85671b0-c3c7-4abe-b814-9464cd9e8ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:50.416903 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85671b0-c3c7-4abe-b814-9464cd9e8ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:50.418374 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85671b0-c3c7-4abe-b814-9464cd9e8ccb service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:50.427504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1453/2905] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:54.334571 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:54.394531 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25032f38-f763-4cb3-85bf-2e19e5b3b145 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:54.399433 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:54.400088 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:54.400362 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:54.400631 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] PATH_INFO: `/v3/users/8fc7add8f1b24eb8ba1fd6176573a61d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:54.401722 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc7add8f1b24eb8ba1fd6176573a61d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:54.403651 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:54.408091 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:54.448587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc7add8f1b24eb8ba1fd6176573a61d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:54.455807 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc7add8f1b24eb8ba1fd6176573a61d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:54.468422 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc7add8f1b24eb8ba1fd6176573a61d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:54.469067 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc7add8f1b24eb8ba1fd6176573a61d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:54.476708 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc7add8f1b24eb8ba1fd6176573a61d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:54.483211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1453/2906] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:54 2022] DELETE /identity/v3/users/8fc7add8f1b24eb8ba1fd6176573a61d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:54.488031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:54.488539 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:54.488684 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:54.507590 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:54.508763 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:54.565325 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1454/2907] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:54.575413 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:54.576833 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:54.632645 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ca05-234f-4d37-9bbf-a548592e32f6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:54.636221 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:54.636813 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:54.637096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:54.637368 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:54.646117 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:54.647529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:54.657141 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1454/2908] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:12:54.917433 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:54.919256 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:54.933593 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715f346-89b9-44aa-be65-b0438ada4243 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:54.938631 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:54.939366 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:54.939709 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:54.940142 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] PATH_INFO: `/v3/projects/75de5634834e4dbeaed305d39608d86e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:54.941459 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75de5634834e4dbeaed305d39608d86e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:54.943911 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:54.960492 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:54.990654 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75de5634834e4dbeaed305d39608d86e', 'request_id': 'req-f51efcd8-4ae8-4c9f-816b-b040092555f3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:54.996916 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75de5634834e4dbeaed305d39608d86e', 'request_id': 'req-f51efcd8-4ae8-4c9f-816b-b040092555f3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.002907 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75de5634834e4dbeaed305d39608d86e', 'request_id': 'req-f51efcd8-4ae8-4c9f-816b-b040092555f3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.015349 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] The token cache is being invalidate because project 75de5634834e4dbeaed305d39608d86e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:55.015641 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.016337 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.017887 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1455/2909] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:54 2022] DELETE /identity/v3/projects/75de5634834e4dbeaed305d39608d86e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:55.021846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:55.031333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df099ad8-c0d9-47ac-8b9e-2a19be86f5c1 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:55.086209 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:55.086874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:55.087146 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:55.087393 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] PATH_INFO: `/v3/users/96553802c82047979e12215dda20c2f9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:55.088655 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96553802c82047979e12215dda20c2f9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:55.090677 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:55.119926 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:55.144096 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96553802c82047979e12215dda20c2f9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.151466 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96553802c82047979e12215dda20c2f9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.163177 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96553802c82047979e12215dda20c2f9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.163794 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96553802c82047979e12215dda20c2f9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.170561 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96553802c82047979e12215dda20c2f9'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.176506 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1455/2910] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:55 2022] DELETE /identity/v3/users/96553802c82047979e12215dda20c2f9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:55.437837 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:55.439487 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:55.453307 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51efcd8-4ae8-4c9f-816b-b040092555f3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:55.509356 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:55.510027 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:55.510283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:55.510557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] PATH_INFO: `/v3/projects/666dc42a35cb4008aa91d31209a5da20` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:55.511679 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] RBAC: Authorizing `identity:delete_project(project_id=666dc42a35cb4008aa91d31209a5da20)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:55.513576 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:55.561093 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666dc42a35cb4008aa91d31209a5da20', 'request_id': 'req-f3663721-3262-4c04-b09e-155e3fdbf91c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.566630 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666dc42a35cb4008aa91d31209a5da20', 'request_id': 'req-f3663721-3262-4c04-b09e-155e3fdbf91c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.572791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666dc42a35cb4008aa91d31209a5da20', 'request_id': 'req-f3663721-3262-4c04-b09e-155e3fdbf91c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.584779 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] The token cache is being invalidate because project 666dc42a35cb4008aa91d31209a5da20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:55.585070 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.585681 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.587156 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1456/2911] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:55 2022] DELETE /identity/v3/projects/666dc42a35cb4008aa91d31209a5da20 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:55.591375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:55.592765 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:55.605599 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46162c-fde3-444c-adc3-5a7d8ff04c27 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:55.660035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:55.660587 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:55.660874 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:55.661152 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] PATH_INFO: `/v3/users/d9962d5891a14d5f830c830a8504dffc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:55.662283 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9962d5891a14d5f830c830a8504dffc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:55.664803 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:55.707218 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9962d5891a14d5f830c830a8504dffc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.714797 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9962d5891a14d5f830c830a8504dffc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.726846 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9962d5891a14d5f830c830a8504dffc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.727472 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9962d5891a14d5f830c830a8504dffc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.734125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1da19065-9770-4385-b29b-539396083e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9962d5891a14d5f830c830a8504dffc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:55.739819 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1456/2912] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:12:55 2022] DELETE /identity/v3/users/d9962d5891a14d5f830c830a8504dffc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:56.001764 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.003239 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:56.019031 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3663721-3262-4c04-b09e-155e3fdbf91c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.050803 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:56.078822 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.079455 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.079739 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.080011 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] PATH_INFO: `/v3/projects/5216685e5d304d52a24c1e08e419ac1c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.081147 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5216685e5d304d52a24c1e08e419ac1c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.083136 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.128958 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5216685e5d304d52a24c1e08e419ac1c', 'request_id': 'req-f13ec4f9-f205-496b-a14f-07896ef370eb'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:56.134814 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5216685e5d304d52a24c1e08e419ac1c', 'request_id': 'req-f13ec4f9-f205-496b-a14f-07896ef370eb'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:56.140694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5216685e5d304d52a24c1e08e419ac1c', 'request_id': 'req-f13ec4f9-f205-496b-a14f-07896ef370eb'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:56.152855 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] The token cache is being invalidate because project 5216685e5d304d52a24c1e08e419ac1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:12:56.153139 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:56.153748 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f13ec4f9-f205-496b-a14f-07896ef370eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:12:56.155252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1457/2913] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:12:56 2022] DELETE /identity/v3/projects/5216685e5d304d52a24c1e08e419ac1c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:56.160375 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.160668 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.160946 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.182772 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:56.184165 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:56.203160 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:56.243768 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1457/2914] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:56.252893 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.253183 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.253464 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.273851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:56.275098 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:56.282926 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1458/2915] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:56.287005 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.296556 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4db1f-179c-4c60-8110-5342dd13b4d3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.301166 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.301653 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.301991 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.302270 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.302911 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 15:12:56.303211 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.304787 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.316014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1458/2916] 10.222.0.61 () {58 vars in 1153 bytes} [Sat Sep 24 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 15:12:56.320626 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.329565 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f0604-a72d-4068-b63a-6fc89bfcb334 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.334234 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.334820 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.335713 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.335980 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.336588 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.338549 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.364082 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1459/2917] 10.222.0.61 () {60 vars in 1153 bytes} [Sat Sep 24 15:12:56 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 15:12:56.368385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.378029 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcc81b-44b0-479e-be20-303f6bd24b71 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.434066 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.434579 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.434858 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.435128 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.435755 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.437667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.459044 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1459/2918] 10.222.0.61 () {60 vars in 1144 bytes} [Sat Sep 24 15:12:56 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 15:12:56.463133 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.473188 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fdb88-acca-41aa-8fc6-fcaa1094337b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.477807 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.478426 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.478704 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.478970 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.479599 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.481215 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.492091 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1460/2919] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:56.495827 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.505398 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c982cf5-1641-4977-b9dc-c63501409778 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.510515 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e90f3-dfdb-480f-b47b-4334ae452556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.511092 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e90f3-dfdb-480f-b47b-4334ae452556 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.511372 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e90f3-dfdb-480f-b47b-4334ae452556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.511642 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e90f3-dfdb-480f-b47b-4334ae452556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.512271 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e90f3-dfdb-480f-b47b-4334ae452556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.513838 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e90f3-dfdb-480f-b47b-4334ae452556 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.524225 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1460/2920] 10.222.0.61 () {58 vars in 1122 bytes} [Sat Sep 24 15:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 15:12:56.528173 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.538101 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009315f5-c2a8-4b6a-b973-bfc15778b2a4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.542657 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.543292 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.543573 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.543844 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] PATH_INFO: `/v3/projects/5ade3f2db3ce4ff78a2e04a9fb1514cf/users/7a19de50fb5c4558b1e9558d30245fb2/roles/0f6c315ba2fd4e86ae0ef609c5a92319` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.558355 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ade3f2db3ce4ff78a2e04a9fb1514cf, user_id=7a19de50fb5c4558b1e9558d30245fb2, role_id=0f6c315ba2fd4e86ae0ef609c5a92319)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.561037 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.574504 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1461/2921] 10.222.0.61 () {60 vars in 1483 bytes} [Sat Sep 24 15:12:56 2022] PUT /identity/v3/projects/5ade3f2db3ce4ff78a2e04a9fb1514cf/users/7a19de50fb5c4558b1e9558d30245fb2/roles/0f6c315ba2fd4e86ae0ef609c5a92319 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:12:56.584914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.585203 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.585480 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.612333 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] MFA Rules not processed for user `7a19de50fb5c4558b1e9558d30245fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:12:56.613520 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:56.675566 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1461/2922] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:12:56.687902 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:12:56.689499 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:12:56.752694 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:12:56.767707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d303ec-6362-4164-b5d3-5211961746b3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:12:56.771399 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44434de1-f19f-4d45-b32c-a29ec271e5da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f49af0c95b458291e68167d99199f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:12:56.771998 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434de1-f19f-4d45-b32c-a29ec271e5da service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:12:56.772279 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434de1-f19f-4d45-b32c-a29ec271e5da service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:12:56.772555 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44434de1-f19f-4d45-b32c-a29ec271e5da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:12:56.780731 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434de1-f19f-4d45-b32c-a29ec271e5da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:12:56.782184 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44434de1-f19f-4d45-b32c-a29ec271e5da service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:12:56.795347 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1462/2923] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:12:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:13:06.672661 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:06.787127 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=8 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:23.537125 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:23.598267 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f726ba-26e9-405b-8332-46cab34e4c6c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:23.603972 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:23.604691 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:23.605085 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:23.606526 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] PATH_INFO: `/v3/users/7a19de50fb5c4558b1e9558d30245fb2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:23.609200 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a19de50fb5c4558b1e9558d30245fb2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:23.611625 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:23.656847 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a19de50fb5c4558b1e9558d30245fb2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:23.663917 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a19de50fb5c4558b1e9558d30245fb2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:23.675871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a19de50fb5c4558b1e9558d30245fb2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:23.676494 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a19de50fb5c4558b1e9558d30245fb2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:23.683252 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a19de50fb5c4558b1e9558d30245fb2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:23.688914 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1462/2924] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:13:23 2022] DELETE /identity/v3/users/7a19de50fb5c4558b1e9558d30245fb2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:13:23.693367 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173c635-d716-4f6d-8156-c826270aadae None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:23.693888 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173c635-d716-4f6d-8156-c826270aadae None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:23.694034 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173c635-d716-4f6d-8156-c826270aadae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:23.713283 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d173c635-d716-4f6d-8156-c826270aadae None None] MFA Rules not processed for user `3ea61b3146144581bb15be72dcc836fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 15:13:23.714485 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d173c635-d716-4f6d-8156-c826270aadae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:23.770432 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1463/2925] 10.222.0.61 () {58 vars in 958 bytes} [Sat Sep 24 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 15:13:23.780409 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:23.781804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:23.839285 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc8786-fe56-4927-833f-da73ecd48815 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:23.842871 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:23.843461 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:23.843738 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:23.844014 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:23.853222 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:23.854646 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:23.863694 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1463/2926] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:13:24.118696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173c635-d716-4f6d-8156-c826270aadae None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:24.120119 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d173c635-d716-4f6d-8156-c826270aadae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:24.132112 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173c635-d716-4f6d-8156-c826270aadae None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:24.137508 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:24.138181 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:24.138466 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:24.138758 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] PATH_INFO: `/v3/projects/5ade3f2db3ce4ff78a2e04a9fb1514cf` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:24.139878 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ade3f2db3ce4ff78a2e04a9fb1514cf)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:24.141808 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:24.187278 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ade3f2db3ce4ff78a2e04a9fb1514cf', 'request_id': 'req-7e68bcdb-2ed6-425e-8041-36952398f196'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:24.192799 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ade3f2db3ce4ff78a2e04a9fb1514cf', 'request_id': 'req-7e68bcdb-2ed6-425e-8041-36952398f196'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:24.199364 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ade3f2db3ce4ff78a2e04a9fb1514cf', 'request_id': 'req-7e68bcdb-2ed6-425e-8041-36952398f196'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:24.211117 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] The token cache is being invalidate because project 5ade3f2db3ce4ff78a2e04a9fb1514cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:13:24.211411 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:24.212040 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:24.213497 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1464/2927] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:13:24 2022] DELETE /identity/v3/projects/5ade3f2db3ce4ff78a2e04a9fb1514cf => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:13:30.943451 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:30.952804 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617d4d-57f4-4517-8a1c-d737f6baf494 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:30.970810 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:31.003182 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:31.003777 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:31.004063 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:31.004338 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:31.006294 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:31.063055 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:31.064469 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:31.075121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1464/2928] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:13:34.208150 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:41.070281 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=38 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:49.146813 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:49.149022 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:49.161179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68bcdb-2ed6-425e-8041-36952398f196 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:49.165442 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84a66b2803614363a196ecd94828a3fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:49.166146 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:49.166431 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:49.166708 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:49.168379 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:49.235851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:49.237304 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:49.247203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1465/2929] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:13:52.293468 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:52.295164 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:52.383723 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c56054-8e73-4548-a258-26300d43b3b9 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:52.389079 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:52.389753 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:52.390224 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:52.390510 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:52.392373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:52.408185 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:52.409786 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:52.422901 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1465/2930] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:13:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:13:55.479743 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:55.481252 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:55.494784 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1c063-b89f-40d4-bff8-0bea0bc236d8 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:55.499557 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:55.500261 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:55.500615 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:55.500976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] PATH_INFO: `/v3/users/5a93a2d9266541e7bff502ba074fef5c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:55.502192 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a93a2d9266541e7bff502ba074fef5c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:55.504203 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:55.551696 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a93a2d9266541e7bff502ba074fef5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.559277 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a93a2d9266541e7bff502ba074fef5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.571603 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a93a2d9266541e7bff502ba074fef5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.572227 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a93a2d9266541e7bff502ba074fef5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.578991 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a93a2d9266541e7bff502ba074fef5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.585369 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1466/2931] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:13:55 2022] DELETE /identity/v3/users/5a93a2d9266541e7bff502ba074fef5c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:13:55.765608 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:55.779037 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4540a5-6d33-42d0-9505-30d7225ad146 service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:55.874929 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:55.875923 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:55.876385 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:55.876853 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] PATH_INFO: `/v3/projects/cd72399e0d4f4e25b992e8e8cf836c18` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:55.878680 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd72399e0d4f4e25b992e8e8cf836c18)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:55.882716 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:55.946584 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd72399e0d4f4e25b992e8e8cf836c18', 'request_id': 'req-bebb6305-4657-41ea-b125-13935e11a333'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.953529 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd72399e0d4f4e25b992e8e8cf836c18', 'request_id': 'req-bebb6305-4657-41ea-b125-13935e11a333'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.960470 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd72399e0d4f4e25b992e8e8cf836c18', 'request_id': 'req-bebb6305-4657-41ea-b125-13935e11a333'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.975788 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] The token cache is being invalidate because project cd72399e0d4f4e25b992e8e8cf836c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:13:55.976314 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.977035 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:55.978667 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1466/2932] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:13:55 2022] DELETE /identity/v3/projects/cd72399e0d4f4e25b992e8e8cf836c18 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:13:55.982635 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:55.984111 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:55.997791 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38775da1-581c-4058-a028-69f693dfe9ee admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:56.053388 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:56.054568 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:56.054851 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:56.055124 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] PATH_INFO: `/v3/users/e3ca8f7b57314a2484f4deeb7caea50a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:56.056266 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ca8f7b57314a2484f4deeb7caea50a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:56.058226 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:56.063515 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:56.107381 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ca8f7b57314a2484f4deeb7caea50a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.114371 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ca8f7b57314a2484f4deeb7caea50a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.126707 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ca8f7b57314a2484f4deeb7caea50a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.127362 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ca8f7b57314a2484f4deeb7caea50a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.134642 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ca8f7b57314a2484f4deeb7caea50a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.140439 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1467/2933] 10.222.0.61 () {58 vars in 1224 bytes} [Sat Sep 24 15:13:55 2022] DELETE /identity/v3/users/e3ca8f7b57314a2484f4deeb7caea50a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:13:56.309274 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:13:56.310766 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:13:56.324305 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb6305-4657-41ea-b125-13935e11a333 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:13:56.342373 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:13:56.388795 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ea61b3146144581bb15be72dcc836fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54d7e41cbbc346c5a0a397fdcad9ecdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:13:56.389421 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:13:56.389703 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:13:56.390012 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] PATH_INFO: `/v3/projects/3c166880b2924c868d69e3abc7e03e7f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:13:56.391121 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c166880b2924c868d69e3abc7e03e7f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:13:56.393013 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:13:56.437892 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c166880b2924c868d69e3abc7e03e7f', 'request_id': 'req-ade294fd-512d-45f2-8cac-b640cc21bc9d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.443191 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c166880b2924c868d69e3abc7e03e7f', 'request_id': 'req-ade294fd-512d-45f2-8cac-b640cc21bc9d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.450030 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c166880b2924c868d69e3abc7e03e7f', 'request_id': 'req-ade294fd-512d-45f2-8cac-b640cc21bc9d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.462089 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] The token cache is being invalidate because project 3c166880b2924c868d69e3abc7e03e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 15:13:56.462379 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.463008 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 15:13:56.464462 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1467/2934] 10.222.0.61 () {58 vars in 1233 bytes} [Sat Sep 24 15:13:56 2022] DELETE /identity/v3/projects/3c166880b2924c868d69e3abc7e03e7f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 15:14:06.137062 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:14:06.459093 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:14:11.092537 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:14:11.093976 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:14:11.162971 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822eb699-2579-4aec-8934-e19194284ca9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:14:11.219836 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:14:11.220437 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:14:11.220716 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:14:11.220987 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:14:11.222719 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:14:11.234100 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:14:11.235662 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16d59d-6ed5-463c-822b-145ca646a0ef service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:14:11.246179 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1468/2935] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:14:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 15:14:21.241354 dv-cinder-859194-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=58 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 15:19:15.084649 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 15:19:15.086111 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 15:19:15.170330 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade294fd-512d-45f2-8cac-b640cc21bc9d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 15:19:15.175406 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab8ef07-3c09-473b-8061-ae9361b7ec72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '359a86b84b994530b005b43c6c711884', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4a786bafe37463aa9b9eb7bbb2b47b8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 15:19:15.176002 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ef07-3c09-473b-8061-ae9361b7ec72 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 15:19:15.176507 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ef07-3c09-473b-8061-ae9361b7ec72 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 15:19:15.176791 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ef07-3c09-473b-8061-ae9361b7ec72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 15:19:15.185969 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ef07-3c09-473b-8061-ae9361b7ec72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 15:19:15.187486 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab8ef07-3c09-473b-8061-ae9361b7ec72 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 15:19:15.197188 dv-cinder-859194-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1468/2936] 10.222.0.61 () {60 vars in 1365 bytes} [Sat Sep 24 15:19:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)